Systems and methods for personalized generalized content recommendations

ABSTRACT

Users receive content recommendations from a personalized, generalized recommendation service that aggregates and selects content of high personal relevance to each individual user from a large pool of both personal and public content. The received content is filtered and the content determined to be relevant is cached. When a user request for content is received, the cached content is rescored and the content determined to be most relevant based on satisfaction of a relevance threshold is selected and forwarded to the user. Feedback methodologies are also implemented so that a user&#39;s actions are taken into consideration in real time and can affect subsequent recommendations to the user.

The present disclosure relates to personalized, generalized contentrecommendation systems and methods.

BACKGROUND

The ubiquitous presence of the Internet in the modern era has placed theworld of information and entertainment at our finger tips. Users canaccess vast resources of infotainment at any time with nearly minimumeffort so that almost any need for information can be instantlyanswered. In view of the vast resources available and the busy lifestyles, the users' attention is highly fragmented between disparateinformation sources so that it is difficult for any one informationsource to hold the user's attention for a considerable time period.

SUMMARY

This disclosure provides for personalized, generalized recommendationsystems and methodologies that facilitate determining relevance andrecommending relevant content selected from different public and privatedata. In one embodiment a method of providing content recommendations isdisclosed. This method includes receiving, at a processor, a pluralityof content items obtained from content streams of different contenttypes to be forwarded to at least one user. In an embodiment, thecontent streams of different content types are received from multiplecontent sources. Attributes of the received content items are obtainedand content items to be forwarded to the user are obtained based on theattributes. In an embodiment, the content items determined to berelevant based on the attributes are forwarded to a user modelassociated with the user. The content items forwarded to the user modelare scored and content items determined to be relevant based on thescore are added to the lists of unseen content items. At least a firstsubset of the scored content items are added to respective lists ofunseen content items for the user based at least on the type of contentin each content item of the subset. When a request is received from theuser for relevant unseen content items, the first subset of contentitems are rescored based on a current context of the user. Top scoringcontent items are selected from the first subset of content items andforwarded to the user. In an embodiment, a user selection of a contentitem from the selected top scoring content items is received by theprocessor that recalculates the relevance of the user selected contentitem to user models of other users on receiving the user selection.

In another embodiment, a computing device, comprising a processor and astorage medium for tangibly storing thereon programming logic associatedwith the personalized, generalized, recommendation system, for executionby the processor, is disclosed. The programming logic comprises contentitem receiving logic, executed by the processor, for receiving aplurality of content items obtained from content streams of differentcontent types to be forwarded to at least one user. Scoring logic isexecuted by the processor, for scoring at least a subset of theplurality of content items. Adding logic, executed by the processor,adds at least a first subset of the scored content items to respectivelists of unseen content items for the user based at least on the type ofcontent in each content item of the subset. User request receivinglogic, executed by the processor, configures the processor to receive arequest from the user for relevant unseen content items. The firstsubset of content items are rescored based on a current context of theuser upon receiving the user request by rescoring logic, executed by theprocessor. Selecting logic, executed by the processor, selects topscoring content items from the first subset of content items and theselected top scoring content items are forwarded to the user in responseto the user request for the relevant items by forwarding logic, executedby the processor.

A computer readable storage medium, having stored thereon, instructionsfor execution by a processor is disclosed in accordance with anotherembodiment. The instructions cause the processor to receive a pluralityof content items obtained from content streams of different contenttypes to be forwarded to at least one user and score at least a subsetof the received content items. The instructions further cause theprocessor to add at least a first subset of the scored content items torespective lists of unseen content items for the user based at least onthe type of content in each content item of the subset. When a userrequest for relevant unseen content items is received, the instructionsfurther cause the processor to rescore the first subset of content itemsbased on a current context of the user. In addition, the instructionsalso comprise instructions that configure the processor to select topscoring content items from the first subset of content items and forwardthe selected top scoring content items to the user in response to theuser request.

In one embodiment, a method of providing content is disclosed. Themethod comprises receiving, at a processor, a request from a user for apredetermined number of unseen content items relevant to the currentuser context and transmitting, by the processor, the user request to arecommendation server. A list of seen content items is obtained by theprocessor in response to the user request, and displayed on a displaydevice. The list of seen content items comprises unseen content itemsrelevant to the current user context. In one embodiment, a combinationof the list of seen content items and at least one content itemdisplayed prior to receiving the list of seen content items can bedisplayed.

In an embodiment, a method of generating a user model in terms ofcategories of interest to the user and providing recommendations basedon such a user model is disclosed. The method commences with a processorreceiving, a user request for content recommendations along withinformation regarding the user which can comprise at least a subset ofcontent sources from which the user desires content recommendations. Inan embodiment, the information regarding the user can also compriseinformation associated with prior user activity, such as, previous usersearches, selections of content or prior user feedback. Based on suchuser information, a plurality of categories are provided for selectionto the user. The user selections of categories are received and therespective category vectors of the user selected categories areaggregated. A user model representing the user's interests which isemployed in making relevant content item recommendations is generatedfrom the aggregated category vectors in combination with vectorrepresentations of the user information. In an embodiment, keywords,entities, content item features and other user information can berepresented as vectors which are combined with the aggregated categoryvectors to generate the user model. Based on a determination ofrelevance by the user model, content recommendations are forwarded tothe user. In one embodiment the recommended content items are selectedfrom the subset of content sources wherein each content source providesa respective content type different from other sources.

In one embodiment, the user model is updated based on a user selectionof a recommended content item. This can trigger discovery of new contentand consequently new content recommendations. In an embodiment, suchupdates to user model in response to user selections and recommendationsof new content can occur in real time. Hence, in response to a userselection of a particular content item, updated recommendations ofcontent items can be transmitted to the user. Such updatedrecommendations of content items can comprise recommendations for newcontent items identified by the processor as being relevant to the userin accordance with the updated user model. In an embodiment, updatedrecommendations of content items can be based on updates to other usermodels that are similar to the generated user model. In an embodiment,updated recommendations of content items can be based on updates to thecategory vectors included in the user model.

In an embodiment the user model can be updated on a periodic basis, forexample, on a daily basis. User vectors associated with the user modelover the preceding ‘N’ days are obtained, N being a natural number, forexample, thirty. In a further embodiment, the user vectors of thepreceding thirty days are obtained and are weighed by a function of thenumber of days back and aggregated to generate an aggregated uservector. Current category vectors of the respective categories arecombined with the aggregated user vector to generate an updated uservector for the day.

In one embodiment, the user model parameters such as relevance thresholdare adjusted based on user behavior and/or system response. In oneembodiment, the number of times that a user requests new content isrecorded. If the user requests new content frequently, a relevancethreshold associated with determination of relevance for providing therecommendations is lowered, such that more content items can berecommended. Conversely, if the user requests new content lessfrequently, the relevance threshold is increased such that fewerrelevant content items can be recommended. In one embodiment, types ofcontent requested by the user can be stored and the user model can beupdated such that a greater number of category vectors are aggregatedwithin the user model if the user is requesting greater variety ofcontent or content of different content types.

In one embodiment, a computing device for generating a user model interms categories of interest to the user and providing recommendationsbased on such a user model is disclosed. The computing device comprisesa processor and a storage medium for tangibly storing thereon programlogic for execution by the processor. The program logic when executed bythe processor, causes the processor to receive a user request forcontent recommendations in addition to receiving information regardingthe user comprising at least a subset of content sources from which theuser desires content recommendations. The processor also executes logicfor providing categories for selection by the user and logic forreceiving category selections made by the user. Aggregating logic,executed by the processor, aggregates category vectors of respectivecategories selected by the user. A user model representing the user'sinterests is generated by the generating logic executed by the processorwhich generates the user model by combining the aggregated categoryvectors with vector representations of the user information.Recommendations providing logic is executed by the processor, forproviding recommendations of content items to the user based on the usermodel. In one embodiment, the recommendations providing logic furthercomprises, logic for selecting the recommended content items from thesubset of content sources, wherein each content source provides arespective content type different from other sources. The processor alsoexecutes logic for receiving user selection of at least one recommendedcontent item and updating logic, for updating the user model based onthe user selection. In addition, the processor also executes logic forupdating the recommendations of content items based on the updated usermodel and transmitting the updated recommendations of content items tothe user in real-time.

In one embodiment, the processor also executes logic for providing,updated recommendations of content items based on updates to other usermodels that are similar to the generated user model. In an embodiment,the processor executes logic for providing updated recommendations ofcontent items based on updates to the category vectors included in theuser model. In an embodiment, the processor executes logic for updatingthe user model on a periodic basis by obtaining, user vectors associatedwith the user model over prior ‘N’ days, N being a natural number,aggregating, the user vectors of the preceding days weighed by afunction of the number of days back to generate an aggregated uservector and by combining, current category vectors of the respectivecategories with the aggregated user vector to generate an updated uservector.

In an embodiment, the processor executes storing logic, for storingfrequency of user requests for new content so that the processor canlower a relevance threshold associated with determination of relevancefor providing the recommendations such that more content items can berecommended if the user request new content frequently. Conversely, theprocessor can increase the relevance threshold such that fewer relevantcontent items can be recommended if the user requests new contentinfrequently. In an embodiment, the processor executes logic for storingtypes of content requested by the user and updating, the user model suchthat a greater number of category vectors are aggregated within the usermodel.

In an embodiment, the processor executes explicit training logic, forreceiving terms from the user for explicitly adding to or deleting fromthe user model and item weighing logic, for respectively weighingfavorably and unfavorably content items comprising the received termswhen providing the recommendations. Additionally, the processor alsoexecutes training logic, for suggesting terms from the content items tothe user for adding to the user model and/or for content searches thatmay be issued by the user. In an embodiment, the processor executes usermodel comparison logic, for comparing the user model to a disparate usermodel of a second user. Contact suggestion logic, executed by theprocessor, suggests the second user as a contact to the user based on asimilarity of the user model to the disparate user model of the seconduser as determined by the user model comparison logic. For example, thecontact suggestion logic can be activated by the processor, based on thesimilarity between the two user models crossing a predeterminedthreshold value.

In one embodiment, a computer readable storage medium, having storedthereon, instructions which when executed by a processor, cause theprocessor to provide relevant content recommendations is disclosed.These include instructions that cause the processor to receive, a userrequest for content recommendations in addition to information regardingthe user comprising at least a subset of content sources from which theuser desires content recommendations. The instructions cause theprocessor to provide categories for selection by the user and receive,category selections made by the user. The instructions further includethose that cause the processor to aggregate category vectors ofrespective categories selected by the user and generate a user modelrepresenting the user's interests. In an embodiment, the user model isgenerated from the aggregated category vectors in combination withvector representations of the user information and recommendations ofcontent items to the user are based on the user model. In an embodiment,the user information can include information associated with useractivity prior to generation of the user model. In an embodiment, thecomputer readable medium also includes instructions to select therecommended content items from the subset of content sources, eachcontent source providing a respective content type different from othersources of the subset of content sources. In one embodiment, thecomputer readable storage medium, further comprises instructions tostore the recommended content items in a data storage and generate anindex on the stored content items that facilitates later user retrievalof the stored content items from the data storage. Relevant contentitems are retrieved from the stored content items in response to a userquery and ranked based on prior user actions associated therewith. Theranked content items are transmitted in response to the user query.

A method for providing content recommendations is disclosed inaccordance with one embodiment. The method comprises, receiving, at aprocessor, a user request for content recommendations. A plurality ofcontent categories for user selection are provided, by the processor. Atleast, a subset of the categories selected by the user are received andbased at least on such user selected categories a subset of avatars aregenerated for further selection by the user. Upon receiving a userselection of an avatar from the subset of avatars, the user selectedavatar is associated with a respective user model of the user. The usermodel can be customized by exchanging communication with the user viathe avatar. In an embodiment, the user model can be customized byreceiving search terms for whitelisting from the user via exchange ofmessages between the user and the avatar and generating, standing searchqueries for each of the whitelisted terms. User feedback to contentretrieved via the standing search queries can be monitored andcontextual messages can be provided to the user via the avatar based onthe user feedback. The standing search queries are updated based on userresponse to the contextual messages. In an embodiment, a health of theavatar is indicated based at least on interaction of the user with theuser model wherein the health of the avatar is indicated by theprocessor via a color of the avatar, via an expression of the avatar orsome other visual representation. In an embodiment, a sentiment of acontent item is indicated via an expression of the avatar. In anembodiment, an indication of personalized content can be indicated via arendering of the avatar.

A computing device comprising a processor and a storage medium fortangibly storing thereon program logic for execution by the processorare disclosed in one embodiment. The program logic comprises requestreceiving logic, executed by a processor, for receiving a user requestfor content recommendations and providing logic, executed by theprocessor, for providing a plurality of content categories for userselection in response to the user request. Category selection receivinglogic is executed by the processor for receiving at least a subset ofthe categories selected by the user. Based at least on user selectedcategories, avatar generating logic, executed by the processor,generates a subset of avatars for selection by the user. Avatarselection receiving logic, executed by the processor, receives a userselection of an avatar from the subset of avatars and associating logic,executed by the processor, associates the avatar to a respective usermodel of the user. Additionally, customizing logic is executed by theprocessor, for customizing the user model by exchanging communicationwith the user via the avatar.

A method of providing content recommendations is disclosed in accordancewith one embodiment. The method commences with a processor transmittinga user request for content recommendations. The processor receives aplurality of content categories for user selection wherein at least asubset of the categories are selected by the user. A subset of avatarsfor selection by the user are displayed wherein the subset of avatarsdisplayed to the user are indicative of the user selected categories. Auser selection of an avatar from the subset of avatars is received andcommunication from between user model and the user is facilitated viathe avatar, wherein the user model is associated with the avatar forexample, by displaying messages from the user model to the user.

A computing device comprising a processor and a storage medium fortangibly storing thereon program logic for execution by the processor isdisclosed in accordance with another embodiment. The storage mediumcomprises transmitting logic, executed by a processor, for transmittinga user request for content recommendations. Content category receivinglogic is executed by the processor, for receiving a plurality of contentcategories for user selection and user category selection receivinglogic, executed by the processor, for receiving at least a subset of thecategories selected by the user. Avatars set display logic is executedby the processor, for displaying a subset of avatars for selection bythe user wherein, the subset of avatars are indicative of the userselected categories. Avatar selection receiving logic, executed by theprocessor receives a user selection of an avatar from the subset ofavatars so that communication display logic, executed by the processor,can facilitate communication from a user model to the user via theavatar, wherein the user model is associated with the avatar.

These and other embodiments and embodiments will be apparent to those ofordinary skill in the art by reference to the following detaileddescription and the accompanying drawings.

BRIEF DESCRIPTION OF THE DRAWINGS

In the drawing figures, which are not to scale, and where like referencenumerals indicate like elements throughout the several views:

FIG. 1 is a schematic diagram illustrating a process of providingpersonalized, generalized recommendations in accordance with anembodiment of the present disclosure;

FIG. 2 is a block diagram depicting certain modules within therecommendation system in accordance with an embodiment of the presentdisclosure;

FIG. 3 is a block diagram depicting certain modules within thepersonalized, generalized recommendations module in accordance with anembodiment of the present disclosure;

FIG. 4 shows the user processing element (user PE) that represents auser in accordance with one embodiment;

FIG. 5 illustrates a global user processing element (user PE) inaccordance with an embodiment;

FIG. 6 shows a flowchart illustrating an embodiment of a method offorwarding content items to users in accordance with one embodiment;

FIG. 7 shows a flowchart illustrating an embodiment of a method offorwarding relevant content items to a user associated with a user PEthat receives a new item of content in accordance with one embodiment;

FIG. 8 shows a flowchart illustrating a method of forwarding relevantcontent items to a user in accordance with an embodiment of the presentdisclosure;

FIG. 9 is a schematic figure showing the processes executed by a user PEin accordance with one embodiment;

FIG. 10 is a schematic diagram that illustrates a procedure of updatingthe user PE on a periodic basis in accordance with one embodiment;

FIG. 11A shows a flow chart illustrating a method of initiating a usermodel in accordance with one embodiment;

FIG. 11 B shows a flowchart illustrating an embodiment of a method ofupdating a user vector on a periodic basis;

FIG. 12A shows a flowchart illustrating a method of updating userrecommendations based on user feedback in accordance with an embodiment;

FIG. 12B shows a flowchart illustrating an embodiment of a method ofupdating user recommendations based on user feedback;

FIG. 13 is a flow chart illustrating an embodiment of a method ofproviding user recommendations based on similarity between user models;

FIG. 14 is a flow chart illustrating a method of updating userrecommendations in real time in response to received user feedback inaccordance with an embodiment of the present disclosure;

FIG. 15A shows a methodology that details the procedure of generatingthe avatar in accordance with an embodiment of the present disclosure;

FIG. 15B shows a methodology that details the procedure of generatingthe avatar in accordance with an embodiment of the present disclosure;

FIG. 16 depicts an illustration wherein the avatar employed to act as ahelp agent in accordance with an embodiment of the present disclosure;

FIG. 17 is depicts a schematic diagram of an avatar being employed whenthe user model is operating in a training mode in accordance with anembodiment of the present disclosure;

FIG. 18a shows an avatar informing the user about how it can becomehealthy and the various levels of health in accordance with anembodiment of the present disclosure;

FIG. 18b shows various facial expressions associated with an avatar inaccordance with embodiments of the present disclosure;

FIG. 19 shows a depiction of an embodiment wherein various expressionsof the avatar reflect various emotions in response to different contentin accordance with an embodiment of the present disclosure;

FIG. 20A depicts an embodiment wherein the avatar is rendered on anonline content being viewed by a user;

FIG. 20B depicts an embodiment wherein the avatar is rendered on anonline content being viewed by a user;

FIG. 21, internal architecture of a computing device includes one ormore processing units (also referred to herein as CPUs), which interfacewith at least one computer bus in accordance with one embodiment of thepresent disclosure;

FIG. 22 is a schematic diagram illustrating a client deviceimplementation of a computing device in accordance with embodiments ofthe present disclosure.

DESCRIPTION OF EMBODIMENTS

Subject matter will now be described more fully hereinafter withreference to the accompanying drawings, which form a part hereof, andwhich show, by way of illustration, specific example embodiments.Subject matter may, however, be embodied in a variety of different formsand, therefore, covered or claimed subject matter is intended to beconstrued as not being limited to any example embodiments set forthherein; example embodiments are provided merely to be illustrative.Likewise, a reasonably broad scope for claimed or covered subject matteris intended. Among other things, for example, subject matter may beembodied as methods, devices, components, or systems. Accordingly,embodiments may, for example, take the form of hardware, software,firmware or any combination thereof (other than software per se). Thefollowing detailed description is, therefore, not intended to be takenin a limiting sense.

In the accompanying drawings, some features may be exaggerated to showdetails of particular components (and any size, material and similardetails shown in the figures are intended to be illustrative and notrestrictive). Therefore, specific structural and functional detailsdisclosed herein are not to be interpreted as limiting, but merely as arepresentative basis for teaching one skilled in the art to variouslyemploy the disclosed embodiments.

The present invention is described below with reference to blockdiagrams and operational illustrations of methods and devices to selectand present media related to a specific topic. It is understood thateach block of the block diagrams or operational illustrations, andcombinations of blocks in the block diagrams or operationalillustrations, can be implemented by means of analog or digital hardwareand computer program instructions. These computer program instructionscan be provided to a processor of a general purpose computer, specialpurpose computer, ASIC, or other programmable data processing apparatus,such that the instructions, which execute via the processor of thecomputer or other programmable data processing apparatus, implements thefunctions/acts specified in the block diagrams or operational block orblocks.

In some alternate implementations, the functions/acts noted in theblocks can occur out of the order noted in the operationalillustrations. For example, two blocks shown in succession can in factbe executed substantially concurrently or the blocks can sometimes beexecuted in the reverse order, depending upon the functionality/actsinvolved. Furthermore, the embodiments of methods presented anddescribed as flowcharts in this disclosure are provided by way ofexample in order to provide a more complete understanding of thetechnology. The disclosed methods are not limited to the operations andlogical flow presented herein. Alternative embodiments are contemplatedin which the order of the various operations is altered and in whichsub-operations described as being part of a larger operation areperformed independently.

For the purposes of this disclosure the term “server” should beunderstood to refer to a service point which provides processing,database, and communication facilities. By way of example, and notlimitation, the term “server” can refer to a single, physical processorwith associated communications and data storage and database facilities,or it can refer to a networked or clustered complex of processors andassociated network and storage devices, as well as operating softwareand one or more database systems and applications software which supportthe services provided by the server.

For the purposes of this disclosure a “network” should be understood torefer to a network that may couple devices so that communications may beexchanged, such as between a server and a client device or other typesof devices, including between wireless devices coupled via a wirelessnetwork, for example. A network may also include mass storage, such asnetwork attached storage (NAS), a storage area network (SAN), or otherforms of computer or machine readable media, for example. A network mayinclude the Internet, one or more local area networks (LANs), one ormore wide area networks (WANs), wire-line type connections, wirelesstype connections, cellular or any combination thereof. Likewise,sub-networks, such as may employ differing architectures or may becompliant or compatible with differing protocols, may interoperatewithin a larger network. Various types of devices may, for example, bemade available to provide an interoperable capability for differingarchitectures or protocols. As one illustrative example, a router mayprovide a link between otherwise separate and independent LANs.

A communication link or channel may include, for example, analogtelephone lines, such as a twisted wire pair, a coaxial cable, full orfractional digital lines including T1, T2, T3, or T4 type lines,Integrated Services Digital Networks (ISDNs), Digital Subscriber Lines(DSLs), wireless links including satellite links, or other communicationlinks or channels, such as may be known to those skilled in the art.Furthermore, a computing device or other related electronic devices maybe remotely coupled to a network, such as via a telephone line or link,for example.

A computing device may be capable of sending or receiving signals, suchas via a wired or wireless network, or may be capable of processing orstoring signals, such as in memory, as physical memory states, and may,therefore, operate as a server. Thus, devices capable of operating as aserver may include, as examples, dedicated rack-mounted servers, desktopcomputers, laptop computers, set top boxes, integrated devices combiningvarious features, such as two or more features of the foregoing devices,or the like. Servers may vary widely in configuration or capabilities,but generally a server may include one or more central processing unitsand memory. A server may also include one or more mass storage devices,one or more power supplies, one or more wired or wireless networkinterfaces, one or more input/output interfaces, or one or moreoperating systems, such as Windows Server, Mac OS X, Unix, Linux,FreeBSD, or the like.

Throughout the specification and claims, terms may have nuanced meaningssuggested or implied in context beyond an explicitly stated meaning.Likewise, the phrase “in one embodiment” as used herein does notnecessarily refer to the same embodiment and the phrase “in anotherembodiment” as used herein does not necessarily refer to a differentembodiment. It is intended, for example, that claimed subject matterinclude combinations of example embodiments in whole or in part. Ingeneral, terminology may be understood at least in part from usage incontext. For example, terms, such as “and”, “or”, or “and/or,” as usedherein may include a variety of meanings that may depend at least inpart upon the context in which such terms are used. Typically, “or” ifused to associate a list, such as A, B or C, is intended to mean A, B,and C, here used in the inclusive sense, as well as A, B or C, here usedin the exclusive sense. In addition, the term “one or more” as usedherein, depending at least in part upon context, may be used to describeany feature, structure, or characteristic in a singular sense or may beused to describe combinations of features, structures or characteristicsin a plural sense. Similarly, terms, such as “a,” “an,” or “the,” again,may be understood to convey a singular usage or to convey a pluralusage, depending at least in part upon context. In addition, the term“based on” may be understood as not necessarily intended to convey anexclusive set of factors and may, instead, allow for existence ofadditional factors not necessarily expressly described, again, dependingat least in part on context.

The information age has made available large amounts of data for usersto access via different modalities. This data includes both public orprivate data which is available for access via a computer, telephone,television or any other systems which can include informationprocessors. As more and more personal data goes online and into thecloud, content consumption and other online user activity takes place ina large number of different places and via various devices. A user mayemploy dozens of different web sites and applications during the courseof a day, with all of them competing for the user's attention.Therefore, while a user needs information from a variety of contentsources, the sheer volume of information coming from these sources canbe overwhelming for any one person to reasonably process, especiallywithin a short time span as the case may be for many customers. Variousembodiments described herein provide for recommendation systems andmethods that provide recommendations for content from various privateand public sources personalized to a user's context that includes theusers current needs and preferences. A shortlist of information itemscan be created that the user might need to know here and now.Alternately, the embodiments provide for a “mother of all feeds” thataggregates and forwards, in real time, relevant, actionable and/orengaging content that matters most to the user.

Turning now to the figures, FIG. 1 is a schematic diagram 100illustrating a process of providing personalized, generalizedrecommendations in accordance with one embodiment. A plurality of inputcontent streams or feeds 110 from a plurality of sources 120 arereceived. In an embodiment, the plurality of content sources can includeany constantly streaming source of information. Content from the inputstreams or feeds 110 is aggregated and the aggregated content 130 isfiltered to identify relevant content 140 or content requiring immediateuser attention 140 so that it can be forwarded to the user. Variouscontent sources 120 that produce public and private content can beaccessed and different types of content streams in various data formatscan be aggregated. By the way of illustration and not limitation, thecontent sources 120 can comprise public content sources such as newssources, blogs, deals or alerts such as, those from weather or financialinformation sources. The content sources 120 can also comprise privatecontent sources such as, email, social networks or calendars. Theaggregated content streams 130 can be time-ordered streams filtered byrelevance to the user's current context. In one embodiment, the latestcontent comprised in the aggregated content streams 130 can be displayedto the users. For example, the top ten latest items aggregated from thefeeds 110 can be provided to the user in a personalized web page of theuser or in an application (an “app”) associated with a mobile userdevice. In another embodiment, the aggregated content streams 130 areforwarded to the user only upon receiving a user request for content. Auser request for the latest content from the sources 120 can bereceived. In an embodiment, content that is relevant for current usercontext is retrieved wherein the relevance of content items in theaggregated content streams 130 is determined in real time upon receivingthe user request. For example, the user can activate a ‘gimme 5’ button150 in order to obtain, for example, the top five recommendationsselected from the aggregated content streams 130. In the embodimentillustrated in FIG. 1, the relevant content 140 forwarded to the userincludes a top scoring item which is a TWITTER feed, followed by twonews items and an email. By the way of illustration and not limitation,the information transmitted to the user can include a summary of theitem 170 followed by a link to the item 160. Therefore, the relevantcontent 140 transmitted to the user comprises items from both public andprivate data streams from which content items are scored and orderedbased on various criteria as will be detailed further infra. In oneembodiment, the relevant content 140 forwarded to the user can belimited to include a specific number of content items. In FIG. 1, therelevant content 140 displayed to the user includes the latest tenthings that demand the user's attention. It may be appreciated that themaximum number of content items is specified herein only by the way ofillustration and not limitation.

In one embodiment, a query-less search procedure can be implementedbased on an understanding of a user's needs in real-time, adapting touser's interests changes, location changes and changes in the time ofthe day by employing a filtering mechanism that is adaptive andpersonalized on a per-user basis. In one embodiment, a search based on auser query for a particular type of content from a particular contentstream or for content items having user-specified attributes fromvarious content streams can be aggregated for presentation to the user.

In order to build the personalized content recommendation system 100wherein the content is constantly changing, hyper-personally relevant,the following are non-limiting examples of preferred considerations tobe addressed:

A. Cold-start: Ensuring that the recommended items are relevant thefirst time a user begins to use the recommendation system 100 in orderto mitigate the users from abandoning usage of the recommendation system100.

B. Implicit model: Automatically building a robust representation of theusers' interests with as little explicit customization as possible sinceusers typically will only invest a small amount of cognitive energy oncustomizing the recommendation system 100.

C. Learning & forgetting: Adjusting the user model based on userbehavior (or absence of behavior).

D. Discovery: Ensuring that the user model does not get “trapped” in alocal optimum, and allows the user to discover content on topics theymay not have previously seen, including trending topics.

E. Transparency: Representing the user model in such a way as to aidinterpretation, as well as supporting a user-friendly view (for example,to let the user know what the model thinks are relevant topics forhim/her).

F. De-duping: Detecting duplicate or near-duplicate content, and beingsmart about when to recommend it and when to not recommend it.

A recommendation system or methodology may address at least partiallythe aforementioned challenges by combining the following features:

1. An Information Retrieval vector-space model to represent users,articles, and categories, which allows for a simple similarity criterionused for both recommendation and duplicate detection.

2. Leveraging lightweight explicit customization to prime the cold-startmodel, while also providing a mechanism to support discovery of newtopics as well as trending topics.

3. Exploiting simple “Rocchio” style feedback in real-time to update themodel based on user behavior.

4. Adapting parameters of the model in real-time to target desired userand system behavior, for example, based on click-through rate of theuser or recommendation rate of the recommendation system.

FIG. 2 is a block diagram depicting certain modules within therecommendation system 200 in accordance with an embodiment of thepresent disclosure. The recommendation system 200 can include a usercomputer/client device 202 or 204 that communicates with a servercomputer 206 over a network 208 such as the Internet. In one embodiment,the server 206 includes a personalized, generalized contentrecommendation module 210 which, in turn, can be communicatively coupledto a feed manager 212 that comprises drivers for each data source fromwhich it receives content streams 110. It may be appreciated that therecommendation module 210 and the feed manager 212 are shown separatelyherein by the way of illustration and that a single computing device canimplement the recommendation module 210 and the feed manager 212. In oneaspect, the feed manager 212 is operable to store the received contentstreams or feeds 110 in a recommendation database 214 and/or to furtherprocess the content streams 110 for efficient distribution. For example,the feed manager 212 can extract data and/or metadata from the contentincluded in the various feeds 110 and normalize the metadata intostandardized format and inject events into the recommendations module210.

The personalized, generalized content recommendation module 210 receivesthe content items 252 from the feed manager 212 and processes them inorder to determine those content items that are relevant to the user. Inan embodiment, the relevance of the received content items 252 can becalculated and a subset of the content items that are determined to berelevant can be stored in unseen items list (not shown) for example, inthe recommendations database 214 prior to being forwarded to the user.The relevance of the received content items 252 can be determined byemploying a user model that incorporates the user attributes andpreferences. The user attributes/preferences can be collected explicitlyvia user input, for example, when the user initially signs up with therecommendation module 210 and/or implicitly via automated learningmechanisms incorporated into the recommendation system 100 as will befurther detailed infra.

In one embodiment, a request 254 for unseen content items that arerelevant to the current user context can be received from a useremploying the user device 204. The user's current context can bedetermined based on the information associated with the user request254, such as, the time at which the request was received or the locationinformation. In an embodiment, the location information associated withthe user request 254 can be determined from the user's current GPScoordinates as obtained from the device 204. In addition, the currentuser context can be also be determined based on social context of theuser and its combinations with other context information available fromdifferent public/private sources such as, for example, the user'scalendar. Based on the current user context information gathered by therecommendation module 210, a relevance score of the items stored in theunseen items lists for the user of the device 204 is determined. Asshown in FIG. 2, a list of seen items 256 is generated by therecommendations module 210 for the user of the client device 204 basedon the determined relevance scores. In particular, the relevance scoresof all the relevant content items from the unseen lists that comprisecontent items associated with various content types are compared and thetop ‘n’ content items are selected for presentation to the user where‘n’ is a natural number. By the way of illustration and not limitation,the received content items 252 can comprise a combination of public andprivate data items such as emails, TWITTER and FACEBOOK feeds,news/weather/stock/deal alerts or other information. Relevant contentitems from such feeds are aggregated and scored upon receiving the userrequest 254 so that the list of seen items 256 that is ultimatelytransmitted to the user can comprise the top five or the top ten contentitems that include a mixture of items selected from the various contentfeeds 110. It may be appreciated that if the requisite number ofrelevant content items are not discovered in response to a user request,the display of the user can be a combination of the latest unseenrelevant items and items which are already viewed by the user. In thiscase, the relevance score can be lowered in accordance with anembodiment that will be detailed infra. In one embodiment, the latestunseen items can be placed at the top of the display regardless of theirrelevance relative to the already viewed items.

In an embodiment, the received content items 252 can be stored in adatabase 214 without indexing as their relevance to the different usersis calculated in real time even as they are received by therecommendations module 210 from the feed manager 212. In one embodiment,the recommendations module 210 can further implement a feedbackmechanism in real time. The recommendations module 210 receives a user'sselections of the items in the seen items list 256 and employs theinformation associated with such selections in filtering content itemsto be forward to that user. Thus, for example, based on the userselections, the seen items list 256 can be refreshed in real time.Additionally, it can also affect, in real-time, the content itemsforwarded to other users related to the user such as those that areconnected to the user in an online social network or those that therecommendation module 210 has determined have similar profiles as theuser making the selections.

In an embodiment, the recommendations module 210 can function as acentralized storage for different types of personalized content consumedby a user or it can store the content items forwarded to the user. Thestored contents can be indexed in order to facilitate search andretrieval of specific content items. In an embodiment, therecommendations module 210 can also store not only content recommendedto the user, it can also store and index content that is underconsideration for recommendation regardless of whether or not it getsrecommended. In this context, the recommendations module 210 can providean uber-index of all of the personally relevant content for a user. Inan embodiment, the recommendations module 210 returns search resultsfrom such stored content which are ranked based on past user actions.When issuing a keyword search against the items indexed by therecommendations module 210, the recommendations module 210 takes intoconsideration the user's past actions/interactions with the items whendetermining their rankings so that, for example, items that have beenread or otherwise provided positive feedback for example, will be rankedhigher.

In an embodiment, some content that is particularly urgent for a usercan be recognized and forwarded to the user so that the recommendationsmodule 210 acts as a notification platform. The content that is pushedcan comprise for example, alerts associated with weather, stock prices,traffic.

In an embodiment, an advertisement server 220 can serve contextsensitive advertisements to be displayed on web pages or mobileapplications associated with the recommendations module 210 based atleast in part upon one or more terms associated with the requesting useras will be detailed further infra. Although the advertisement server 220is shown in this embodiment as located on the same server computer 206as the recommendations module 210, it can be appreciated that this isnot necessary. The advertisement server 220 can also be located with thefeed manager 212 or it can also be located independently on an externalmachine that is disparate from both the server computer 206 and the feedmanager 212. In an embodiment, an “ad server” can comprise a server thatstores online advertisements for presentation to users. “Ad serving”refers to methods used to place online advertisements on websites, inapplications, or other places where users are more likely to see them,such as during an online session or during computing platform use, forexample. Advertising may be beneficial to users, advertisers or webportals if displayed advertisements are relevant to interests of one ormore users. In an embodiment, advertisements can be presented to usersin a targeted audience based at least in part upon predicted userbehavior(s) or user profile information.

FIG. 3 is a block diagram depicting certain modules within thepersonalized, generalized content recommendations module 210 inaccordance with an embodiment of the present disclosure. In anembodiment, the recommendations module 210 comprises three modules, theitem processing module 310, the feature processing module 320 and theuser processing module 330. The personalized, generalized contentrecommendations module 210 is built on top of the S4 platform which canbe used for processing continuous unbounded streams of data. In oneembodiment, a processing module within the recommendation module 210 isoperable to generate a processing element which is in an embodiment, aJAVA object that includes high level data and code and which receivesand generates events on content streams in real-time. The processingelements may be distributed across as many computing nodes as necessary.The processing elements have their memory state checkpointed topersistent storage, to allow easy re-instantiation if for some reasonthey are removed from memory (e.g., if a node goes down, or the systemneeds to be restarted). The processing elements expire after a period ofinactivity and are swapped out of memory to persistent storage, but areswapped back in if they start received messages again. This architectureallows for easy scaling so that as more users and more content streamsare added, enough new nodes are added to the system incorporating therecommendation module 210 to cover the memory footprint of the newusers, items and features. For each active user, the recommendationsmodule 210 qualifies or disqualifies the article/content item as beingrelevant enough and if it passes this gate, a reference to the articleis placed in the user's unseen items list, which in turn is sent to theuser as detailed herein. This process of generating events andcategorizing the content for different users is done in real-time sothat the articles/content items are filtered and supplied to the userseven as they are discovered.

Each time a new piece or item of content is received by therecommendation module 210, a representation of the item or an itemprocessing element 312 of the content item is generated by the itemprocessing module 310. As described supra, the item processing element312 can comprise data 3120 and code 3122 associated with events that anitem processing element 312 is configured to process. The data 3120included in the item processing element 312 can comprise the keywords orfeatures associated with the content item represented by the itemprocessing element 312. In addition, the data 3120 can compriseattributes of the content item which can include, for example, thetitle, abstract, source, author, or location associated with or referredto in the content item, in addition to other characterizing and/orstatistical data. By the way of illustration and not limitation, thestatistical data can include data that determines the popularity of thecontent item which can further comprise the number of users to which thecontent item was determined to be relevant, the number of users whoactually clicked on the content item, the number of users wholiked/disliked the content item or the number of users who performedother action such as storing, deleting or forwarding the content item tosocial contacts. In one embodiment, the item processing element 312 canalso store the statistical data associated with each user action, likethe number of times the content item was saved/deleted/forwarded so thatthe relevance of the content item represented by the item processingelement 312 can be better determined relative toss the users of therecommendations module 210.

In an embodiment, users of the recommendations module 210 arerepresented by the user processing elements 332 in a manner similar tothe content items. Thus, each active user of the recommendations module210 is represented by a respective user processing element. When a newuser registers with the recommendation module 210, information relatedto the user is received and processed by the user processing module 330which generates a representation of the user as the user processingelement 332. As described herein, the user processing element 332comprises data 3320 and code 3322 associated with the user itrepresents. The data 3320 can include an initial user model 3324 that isgenerated from the information provided explicitly by the user, forexample, while signing up to use the recommendation module 210 andimplicit information gathered by observing the user's interaction aswill be detailed further infra. The user model 3324 thus generated isfurther constantly updated based on user interaction with therecommendations module 210. This is achieved by configuring the userprocessing element 332 to listen for actions taken by the user itrepresents, e.g., clicking on a given item or forwarding items tocontacts, voting (“like” or “dig”) for the items or other user actionthat is recorded and can be employed to update the user model 3324.

In addition to the aforementioned modules, the recommendation module 210also includes a feature processing module 320 that maintains acorrespondence between users 254 and the received items of content 252.When an item of content is received, features such as keywords and/ormetadata as described herein regarding the item are extracted, forexample, via natural language processing or other techniques. A featurecan generally include any piece of data derived from an item or a userand used as input to the filtering decision. Features can includeexplicit data of content items such as words, or entities. By the way ofillustration and not limitation, entities can comprise informationassociated with the content item such as a source of the content item,author of the content item, a location, a category or a sentimentassociated with the content item. They can also include implicit datasuch as inherent attributes of a content item, for example, an encodingformat of a video item. For each feature that is extracted, a featureprocessing element 322 comprising data 3220 and code 3226 is generatedby the feature processing module 320. For each of the recognizedfeatures, the feature processing module 320 maintains, for example, inthe data 3220 of the corresponding feature processing element 322, alist of content items 3222 associated with a given feature e.g., a wordor phrase and a list of users 3224 who express interest in the givenfeature. For example, for a feature, such as a word “microhybrids”, thefeature processing module 320 maintains a list of a subset of thereceived content items 252 which include or are otherwise associatedwith the word “microhybrids” and a list of active users of therecommendation module 210 who have either implicitly or explicitlyexpressed interest in the content items that include the term“microhybrids”. The code 3224 can further comprise modules forcommunicating with the item processing module 310 in order to update thelist of items 3222 as new items are received by the recommendationsmodule 210. The code 3226 can also comprise modules for communicatingwith the user processing module 330 in order to update the list of users3224 so that users can be added or removed from the users list 3224. Inan embodiment, only public items such as news items are analyzed by thefeature processing module 320, whereas a user's private items such as,emails, are forwarded directly to the user PE 332 for scoring.

When a new item is received by the recommendations module 210, it isanalyzed by the item processing module 310 in order to generate thecorresponding item processing element 312 and to extract variousfeatures from the newly received item that will be forwarded for furtheranalysis to the feature processing module 320. In different aspects, thefeature processing elements can be generated from both public andprivate items based on, for example, user permissions. For each featureobtained from the received content items 252, the feature processingmodule 320 determines if a feature processing element exists thatcorresponds to each of the features thus obtained. For example, if thereceived feature is associated with the feature processing element 322,the newly received item is added to the list of items 3222 comprisedwithin that feature processing element 322 and the newly received itemis forwarded to all the users in the users list 3224 comprised withinthe feature processing element 322 for further scoring. If a featureprocessing element does not exist for a received feature, it isgenerated by the feature processing module 320 and the users and itemslists are generated and they can be updated as new data is received bythe recommendations module 210. In one embodiment, a feature processingelement can also be generated from user input to the recommendationsmodule 210. In this case, it is assumed that the feature processingelement does not exist within the recommendations module 210 because ifthere was a feature processing element for the user input feature, thenthe user would have been added to the users list of the correspondingfeature processing element. However, in this embodiment, a featureprocessing element is generated from user input and its list of itemsand users is expanded as the feature gains popularity among the users atlarge and is used more often in published/public content. The featureprocessing elements, therefore, act as gates that sort the variouscontent items that need to be distributed to the different users basedon their attributes. The feature processing elements act as go-betweensexecuting a firstpass matching of items with users. They can act as adynamic, always-up-to-date index and thus mitigate the need for scoringeach content item by the user model 3324. In one embodiment, only publicdata such as news items or public blog posts are processed by thefeature processing elements whereas private data such as emails intendedfor a particular user bypass the feature processing elements and aretransmitted directly to the user processing element 332 for scoring.

In accordance with an embodiment the different processing elements aregenerated by modeling the users, categories/features and items ofcontent within the recommendations module 210 as vectors. The featurespace for user, item, and category vectors consists of terms (singlewords or bi-grams) that have been stripped of some special and non-ASCIIcharacters, and stopped (common, non-meaningful words are ignored, andbigrams with one or more stopwords are also ignored). Depending on whatfield of the item they are taken from, they may be lowercased as well.All vectors V are kept normalized by setting V_(i)=1 for all V_(i)>1,any time a new vector is computed (this ensures the highest weightedfeature has a value of 1). Furthermore, the number of non-zero entriesin each vector is limited to some number (e.g., 1000 for user and itemvectors, 10,000 for category vectors). Vectors are also normalized bysetting their lowest-value entries to 0 when this maximum is exceeded.

In an embodiment, every item or article is preferably represented by avector: D_(g, c, d) where the item has a guid (global unique identifier)g, is possibly labeled with a category c, and was published/received onday d. A modified version of tf-idf (term frequency-inverse documentfrequency) weighting is used to populate the item vector. Specifically,term frequency is the sum of occurrences of the term in title or snippet(with unigrams in the title counting twice), and the document frequencyterm is:

${\tanh\left( \frac{TotalDocuments}{500 \cdot {DocFreq}} \right)}.$Additionally, when an item has an author or source, that string is addedas a feature with a weight of 1. All new users are asked to select froma list of preferred categories, for example, from a list of eightpossible categories. The initial user model for user x who has selecteda list of categories C, is simply a term vector that is taken from a setof cold-start vectors: V_(c). These cold-start vectors are computed onan ongoing basis. By the way of illustration and not limitation, at tenminute intervals, the vectors for the hundred most recent items for acategory are summed and added to V_(c), which is then decayed using aconstant that equates to a 0.8 daily multiplier.

In an embodiment, category vectors V_(c) are therefore, used as both theinitial user vector, as well as a component of the evolving user vector.This provides a direct way to give the users reasonable contentrecommendations for the outset thereby addressing the cold startchallenge, with an implicit model that requires little input from theuser. It also directly helps set user expectations by making explicit tothem what types of content they can expect to receive asrecommendations. As the category vectors V_(c) are computedcontinuously, they always represent the recent content in any categorymost strongly, and thus, as part of the user vector, they help make sureusers are able to discover new, trending content thereby ensuring that auser model does not get “trapped” in a local optimum. Furthermore, sincedifferent users may use the recommendations module 210 in differentways. Some users may want hyper-relevance while others may expect morediscovery. An explicit parameter α_(x) as detailed further infra totrade-off how much of the category vectors are used in the user vectorand thereby will allow the recommendations module 210 (or the user) toadjust this property.

FIG. 4 shows the user processing element (user PE) 332 that represents auser in accordance with one embodiment. As discussed supra, the userprocessing element 322 includes data 3320 and code 3322 associated witha particular representation of the user. These aid in determination ofaccurate recommendations for the user represented by the user PE 322.The data 3320 can comprise user attributes such as age, location,demographic information of the user in addition to different itemcollections 410 that are unique to the user who is represented by theuser PE 332. These can include without limitation, collections of seenitems, unseen items, saved items, deleted items, items recommended toother users, liked or disliked items. It may be appreciated that thelisting of item collections 410 for the user PE 332 is by no meansexhaustive and that the item collections 410 can be expanded as moreactions can be executed by the user within the recommendations module210.

The data 3320 also includes user preferences 420 that are collected fromthe user, for example, via a settings screen. These can include, forexample, the privacy setting of the user, the themes to be used for auser interface, the information to be displayed for various screens ofthe user interface and other user preferences. In addition to simplyproviding the stream of recommended content, one embodiment of therecommendations module 210 also provides for “lenses” which are filtersbased on different content attributes. The user can therefore, “drilldown” on specific attributes of recommended content, so they see onlytrending content, content from a particular publisher or author, contentfrom a specified time period in the past or future, content associatedwith a given location, content containing specific keywords, contentbased on sentiment (positive, negative, happy, sad, shocking, etc.) andcontent based on any other meta-data. In an embodiment, such lenses canbe associated with specific content sources under the user preferences420.

Again, it may be appreciated that the user settings are listed herein bythe way of illustration and not limitation and that other usersettings/attributes obtained explicitly or implicitly can be included inthe data 3320 as user settings. In one embodiment, the user settings canalso include the user's avatar selection. As will be described furtherinfra, the avatar of a user is employed to represent the user's usermodel and a user's selection of an avatar can be a feature of the usermodel 3324 which is employed in making personalized, generalizedrecommendations to the user.

The user model 3224, also comprised within the user PE 332, generallyincludes all of those parameters of the recommendations module 210 thatare used on a per-user basis to make the filtering decision for an item.It is employed to score and recommend items to a user that itrepresents. As described supra, the user model 3324 is built using theVector Space Model which is a list of terms (words and phrases) or otherfeatures, with associated weights and modified dot-product for measuringsimilarity. Representing the user in a manner similar to which thecontent items are represented, which is in terms of words and phrases,makes users and items somewhat interchangeable leading to many uses aswill be discussed further herein. Since word-based features are used torepresent the user, parts of the user models can be exposed to the usersthemselves which allows direct user manipulation or otherwise providesinsights which can be useful in improving recommendations made to theusers. In fact, modeling the user on word-based features can also assistthe user with other search activities. For example, when the user isissuing a keyword search (either within or outside of therecommendations module 210), terms from the respective user model can beused as search suggestions or as input to the ranking functions so thatcontent items including terms from the user model 3324 are ranked higherin the result set.

In one embodiment, user model 3324 comprises, among other data, thefollowing parts:

A user vector U

A threshold parameter θ_(x)

An explore/exploit parameter α_(x)

Global feedback parameters γ, δ

An article recommendation rate, recorded in real-time (an observedquantity)

Click through rate (CTR), also recorded in real-time (an observedquantity)

It may be appreciated that other user attributes or item features thatare used to determine relevance of content items to a particular usercan also be part of the user model 3324 in accordance with otherembodiments. As discussed supra, the initial user model for a user x whohas selected a list of categories C, is a term vector that is taken froma set of cold-start vectors V_(c). In an embodiment, the categoryvectors included in a user model 3324 are based on the high-levelcategories, for example, News categories, which are specified as beingof particular interest to a user when the user signs up for therecommendation module 210. The vector representation of each category ismaintained by a real time process based on the content items publishedin that category. A combination of these category vectors based on userpreferences is used as the users' initial user model.

Another source for cold-start models are the user's activities on theaccounts they have linked to the recommendations module 210. Forexample, if a user links their FACEBOOK account or other social networkaccount(s) in order to receive recommendations from their FACEBOOKstream, the recommendations module 210 (depending on what gets exposedby FACEBOOK) can gather more important features (like terms) from thataccount, and add them to the user's model. As an example, words from theuser's posts on FACEBOOK or from the posts of entities they follow, fromtheir profile, from what they have “liked”, can be retrieved in order tobe included as part of the vectors used to build up the user model 3324.In general, any activity on any service the user has linked can providefeatures (not limited to terms) that may be added or removed from theirinitial model. In an embodiment, the user model 3324 can be configuredto include content the user has viewed prior to registering with therecommendations module 210. This content can be obtained for example,via the user's search history and the content, such as search terms forthe user's search history is also aggregated to be included in the usermodel 3324. Thus, in the case of new users for whom a user model hasnever been generated, an initial model is built from initial explicituser input and other explicit or implicit information available viavarious sources so that the possibility of providing poorrecommendations and thereby discouraging the users from using therecommendations module 210 is mitigated. This is the so-called“cold-start” challenge, and it also encompasses the ongoing challenge ofmaking sure that the user model does not get “trapped” in alwaysrecommending the same type of content.

In a typical information retrieval model, the range of values for theelements in a vector are taken from the interval [0 . . . infinity]. Inone embodiment, the user vector U is a special vector called a “Max One”vector, which differs from a typical representation. A list of terms 430along with their respective weights that can be used to determine theuser vector U is shown in accordance with one embodiment. The weights ofthe user vector (which is a “Max One” vector) can be associated with aspecific lower-bound, a parameter called MIN_WEIGHT. Only real-valuedweights between MIN_WEIGHT and 1 are allowed for the “Max One” vector,with any operations that cause the weight to go above 1 will insteadreset it to 1. For example, the vector values can be limited to therange [0 . . . 1]. Any operations that cause the weight to go belowMIN_WEIGHT instead cause the term to be removed from the list. A fixedmost important terms to represent a user's interests and which, if someoperation causes to grow longer, will automatically remove the lowestweighted terms so as to stay within the maximum length. This version ofa vector space model has the effect of naturally supporting two of theabilities required by the recommendation module 210. By limiting themaximum weight to 1, it is ensured that the recommendations module 210never overweighs a given term for a user, which can result inover-emphasizing content about a specific topic. In addition, byautomatically having terms “fall off the end” of the vector, the usermodel 3324 naturally “forgets” terms that do not otherwise getreinforced by the user so that the possibility of entries growingunbounded as a result of feedback is mitigated. For example, a user mayhave read an article about soccer at one time, which does notnecessarily indicate a continued, deeper interest in soccer. In fact, ifthis interest is not reinforced by further user selections of contentitems related to soccer, it is likely that the user was only brieflyinterested in that particular content item. Therefore, assigning aminimum weight to a term and removing terms that go below theMIN_WEIGHT, automatically keeps finite, the list of terms associatedwith the user so that only highly rated terms are maintained in the usermodel 3324 and a user's fleeting or less enduring interest in aparticular category/term is forgotten and not maintained within the usermodel 3324. This mitigates the possibility that content items associatedwith such passing interests are discarded as less relevant and are notforwarded to the user.

The next parameter in the user model 3324 is the threshold parameterθ_(x), which is used to determine if an item is relevant enough to beshown in the user's item list. In addition to the daily updates andfeedback updates to the user vector portion of the model, someparameters are also adjusted dynamically. Specifically, the thresholdparameter θ_(x) is adjusted on a periodic basis to regulate the observedarticle recommendation rate or the number of articles added to theuser's item list per unit time. In one embodiment, the thresholdparameter θ_(x) is adjusted to achieve a substantially optimal item typemixture weights wherein content items from various types of contentstreams like news items, social networking feeds, emails or othercontent types are selected for presentation to the user based onfeedback received from the user. This is done by pre-defining a targetrate, and then periodically adjusting θ_(x) up or down using smallincrements depending on whether the recommendation rate is above orbelow this target. Thus, the threshold parameter θ_(x) for a given useris not a fixed number, but is dynamically adjusted based on feedbackreceived from the user. In an embodiment, the recommendations module 210can track the number times the user requests new content and adjustsθ_(x) up or down accordingly. Thus, the threshold parameter θ_(x) islowered if the user is requesting new content frequently so the morecontent is determined to be relevant to the user thereby fulfilling thefrequent user requests. Conversely, the threshold parameter θ_(x) israised for a user who does not request content frequently so that therecommendations module 210 maintains a more stringent standard ofrelevance for recommending items to such a user. In addition, the nextparameter of the user model, the explore/exploit parameter α_(x) is alsoadjusted in a similar manner, depending on observed click through rateas will be described herein. In one embodiment, a mechanism can beincluded to let users directly manipulate both θ_(x) and α_(x) to beable to control the flow of recommendations by themselves.

As described supra, the initial user model for user x who has selected alist of categories C, is simply a term vector that is taken from a setof cold-start vectors: V_(c) which are computed on an ongoing basis. Inan embodiment, on a daily basis, the user vector is updated orrecomputed as:

$\begin{matrix}{{{\alpha_{x}{\sum\limits_{i = 0}^{29}{w_{i}U_{x,{{today} - i}}}}} + {\left( {1 - \alpha_{x}} \right)V_{C}}}{where}} & {{Ex}.\mspace{14mu}(1)} \\{w_{i} - \frac{\left( {2^{- {({i + 1})}} + \frac{1}{N_{x}}} \right)}{2}} & {{Ex}.\mspace{14mu}(2)}\end{matrix}$and α_(x) is a user-specific parameter which controls the amount ofuser-specific features versus cold-start features; N_(x) is the numberof days of history available for user x, and C is the set of categorypreferences for user x. In an embodiment, this update is done nightly bycombining the user model 3324 of the preceding thirty days and thecurrent cold start categories of vectors included in the user model 3324or when the user changes their category preferences. The term w_(i) isthe weight that will bias more recent user models over older user modelsand 1/Nx is used because in the initial thirty days that the userregisters with the recommendation module 210, thirty snapshots of theuser model are not available of updating the user vector. Thus, at anygiven point in time the user vector is a sum of two components: aweighted sum of the user vector across time (emphasizing recentvectors)—this is the ‘exploit’ component and a weighted sum of categoryvectors across time (also emphasizing recent vectors)—which is the‘explore component’. Therefore, if the user associated whose user vectoris determined in accordance with Expression (1) above, requests contentmore frequently, the user vector is adjusted by lowering α_(x) on theassumption that the user likes the recommended content and therefore therecommendations module 210 is serving the relevant content. On the otherhand if the user does not request content frequently, the user vector isadjusted to be weighed more towards the exploitation part of Expression(1). In real-time the user vector is then adjusted using simple“Rocchio” algorithm as will be described further infra.

In an embodiment, when a user x indicates that item D is relevant (forexample, by clicking on it, or “liking” it), the user vector is updatedas:U₁(x,C)(γU₁(x,C)±δD  Ex. (3)where γ and δ are two global parameters, which, in an embodiment are setas γ=0.99, δ=0.85. In an embodiment, δ takes positive values for allpositive forms of feedback (item clicks, shares, saves, thumb-ups, andmaximizes) and a negative value for negative feedback (deletes andthumb-downs). However, the user model 3324 can be extended to usedifferent δ for different types of feedback, and in accordance withother embodiments it may be different on a per-user basis or may beadjusted dynamically.

In an embodiment, the relevance score of a particular item to aparticular user is normalized, squashed dot product given below inexpression 4:

$\begin{matrix}{r_{x,D} = {\tanh\left( {F \cdot {\frac{U_{x} \cdot D}{{D}_{1}}/0.1}} \right)}} & {{Ex}.\mspace{14mu}(4)}\end{matrix}$where F is a “fudge factor” that depends on the content type of theitem, when the item was published, as well as the number of positive andnegative feedback signals the item has received from the overall userpopulation. The tank is used to ensure the score stays in the [0 . . .1] range, as well as to shape the score and spread out the distribution.In one embodiment, mail items have a slightly different relevance scoreas shown below in expression 5:

$\begin{matrix}{r_{x,D} = {0.1 + {\frac{U_{x} \cdot D}{{D}_{1}}/0.05}}} & {{Ex}.\mspace{14mu}(5)}\end{matrix}$It can be appreciated that all constants have been manually determinedby examining data, and are not necessarily optimal and may vary indifferent embodiments.

Various parameters detailed herein thus aid in optimizing the user model3324. These operational parameters can be adjusted based on observedbehavior of the user and the recommendations module 210. Otherrecommender systems use traditional machine learning techniques to buildclassifiers targeting a prediction of probability of click with the hopeof maximizing click through rate. The recommendations module 210 canmonitor key system metrics on a per-user basis (e.g., click rate,recommendation rate, “gimme” rate, etc.) and then adjust some keyparameters (namely, the threshold used to determine recommendation θ_(x)and the parameter used to mix cold-start with historical models α_(x))based on the user's behavior in real-time. Thus, for example, if theuser is requesting new items often, their threshold θ_(x) is lowered sothey get more items, or if the users are clicking on a lot of items, thecold-start part of their model (the term including 1−α_(x)) is weighedmore heavily in order to give them a more diverse set of items.

The user PE 332 also includes various code 3322 modules that enable itto handle different tasks related to updating the user model 3324 andthereby providing relevant content to the user. One of the variousfunctions that a user PE 332 can execute includes determining acold-start model for the user. As discussed previously, when a userinitially registers to use the recommendations module 210, providingaccurate recommendations to the user can be a challenging task basedonly on the minimum user preferences that are normally collected byrecommender systems. Hence, the recommendations module 210 collectsinformation regarding specific content categories that a user isinterested in and defines the initial user model based on the userselected categories so that the user receives substantially relevantrecommendations even during the initial stages of interaction with therecommendations module 210. Subsequently, the initial user model isupdated and fine tuned based on explicit and implicit feedback from theuser as described in accordance with different embodiments herein.

Another function that the user PE 332 is configured to handle isde-duping, scoring and filtering incoming items. When therecommendations module 210 receives a new content item, it is forwardedto the user PE 332 based on an initial determination of relevanceprovided by the feature PEs associated with the new item. Upon receivingthe new item or a vector representation of the new item, the user PE 332initially determines whether the newly received item is a duplicate of apreviously received content item. This can be determined, for example,using cosine similarity of the received item to items in seen, unseenand deleted caches. If it is determined that the new item is not aduplicate, the user PE 332 proceeds with scoring and filtering the itemin accordance with various relevance criteria associated therewith.

The user PE 332 is also configured to expose certain features of theuser model 3324 to the user for explicit training. By the way ofillustration and not limitation, features for explicit training caninclude selected uni/bigrams that are in the item, biasing towardsuppercased, author/source, words in title. As the recommendation module210 models users in the feature space of words and phrases, parts of theuser model 3324 can be exposed to the user for fine-tuning so that moreaccurate recommendations can be obtained. It may be appreciated thatexplicit training of the user model by the user 3324 can be optional andthat the user model 3324 can implement machine learning techniques toautomatically fine-tune the user model 3324 based on implicit feedbackobtained from different sources including the user actions and/oractions of other users who may share similar interests and hence whohave similar user models.

The user PE 332 is designed to service client requests for contentitems. The user PE 332 can make a determination of relevance each timeit obtains a new article and additionally determine relevance betweenvarious content types upon receiving user request. Based on thepredetermined item type weights associated with each content type, theuser PE assembles a list of seen items to be forwarded to the user inresponse to the user request.

A user PE 332 forwards a list of seen items to a user and records useractions on the content items in the received list. Events generated byuser actions such as, clicking on a particular content item, arereceived by the user PE 332 and employed to update the user model inreal time In addition, the user PE 332 also functions to update the usermodel on a periodic basis in accordance with various embodiments asdetailed further infra.

In an embodiment, the user PE 332 can be used to locate contextsensitive, targeted advertisements for presentation from the user. Forexample, information from the user model 3324 can be employed inidentifying targeted advertisements for presentation to the user alongwith the content recommendations. In an embodiment, different types ofadvertisements which are relevant to information in each of the contentitems in the aggregated content stream 130.

FIG. 5 illustrates a global user processing element (user PE) 500 inaccordance with one embodiment. The global user PE 500 is a hypotheticalentity that is designed to address the cold start challenge associatedwith new users of the recommendation module 210 by aiding in theexploration aspects as detailed herein. In particular, the global userPE comprises models of various categories, for example, news categoriesthat can include business 510, entertainment 520, health 530, or othercategories like world 580. In general, the global user PE 500 maintainsa model each for a finite number of categories associated with variouscontent items that can be received by the recommendations module 210. Incase a new document or content item D is received by the recommendationsmodule 210, a vector representation 540 of the content item D isgenerated and a category associated with the content item D isretrieved. For example, if the content item D is associated with thehealth category 530, the vector representation 502 of the content item Dis added to the health category vector and the new health categoryvector thus generated is decayed as detailed supra. Thus, at any giventime, the global user PE 500 conveys a substantially accuraterepresentation of all the words and phrases in the latest news articlesof a particular category. When a new user registers with therecommendations module 210, the new user selects specific categories forreceiving content items and those specific user-selected categories areincluded in the initial user model thereby addressing the cold startchallenge. Additionally, since the user is also represented by a vector,typical clustering techniques can be used to create groups of relatedusers and aggregate their vectors to create the vector for a new user,again by looking at other attributes they share in common like age, sex,location, News category selections, Avatar selection, or otherselections the user makes when configuring the recommendations module210 for personal use.

Moreover, the updated category vector, for example, the updated healthcategory vector described above is also folded into the user models ofthe users who indicated their interest in the health category. Thus, theuser models of all the users are automatically updated with the newfeatures from the health category 530 in real-time and/or on a periodicbasis.

FIG. 6 shows a flowchart 600 illustrating an embodiment of a method offorwarding content items to users. The method begins at 602 wherein anitem of content is received. At 604, different data and metadataassociated with the received item of content are extracted. The dataassociated with the received item of content can comprise features suchas the terms or keywords associated with the item of content, source andauthor of the item of content, a location referred to or associated withthe item of content, the date of publication of the item of content. Itcan be appreciated that data and metadata in addition to those listedherein and associated with the received item of content can be extractedin accordance with this embodiment. At 606 the received item of contentis processed. As detailed herein, the processing of the content item cancomprise normalizing the metadata associated therewith into astandardized format and triggering an event associated with the receivedcontent item. At 608, an item PE, which is a representation of thereceived item of content is generated. At 610, information associatedwith the received item of content is forwarded to respective featurePEs. In this embodiment, if the data extracted from the item of contentis, for example, keywords to be associated with the content item, thereceived item of content or a representation of the received item ofcontent is forwarded to feature PEs that represent those keywordsextracted from the item of content. At 612, the item of content is addedto the item list under each of the feature PEs that receive the item ofcontent. In one embodiment the feature PEs that receive the item ofcontent can be processing elements that represent various newscategories. By updating the feature PEs with new content items inreal-time even as the content items are received, a real-time processfor constantly maintaining a vector representation of each categorybased on the content items published in that category is implemented.

At 614, the item of content is forwarded to the user PEs associated withthe users in the users list comprised within each of the feature PEs.The users list associated with a feature PE is assembled based onexplicit or implicit user input. For example, the user may haveexplicitly added a particular keyword to the list of terms 430 in theuser PE 322 or the user may have provided an implicit input viaselecting or clicking on articles associated with the keywordcorresponding to the feature PE. Therefore, the user has been includedin the users list of the feature PE corresponding to the keyword. Asdiscussed supra, the feature PE is configured for a first-passestimation of relevance of the received content items to the users sothat the user PEs of each individual user is not overwhelmed withcontent whose relevance is to be determined. The received content ofitem is again scored at the user PEs to determine relevance prior tobeing forwarded to the users as will be detailed further infra.

FIG. 7 shows a flowchart 700 illustrating an embodiment of a method offorwarding relevant content items to a user associated with a user PEthat receives a new item of content. The method commences at 702 whereina plurality of items of content are received by a user PE which is arepresentation of a particular user as detailed herein. In anembodiment, the items of content are received from a feature PE upon adetermination that the content items are of interest to the user becausethe user was included in a users list associated with that particularfeature PE. Thus, if a plurality of content items are received by therecommendations module 210, the feature PEs substantially filter outirrelevant content and forward only content that has at least somedegree of relevance to the user. Therefore, only a subset of contentitems received by the recommendations module 210 from the various datasources are actually forwarded to a user PE for relevancedetermination/scoring. At 704, a content item from the received contentitems is selected and de-duping or a process for identifying duplicatecontent items from multiple content sources is executed at 706. Thus, ifit is determined that the received item of content is a duplicate of apreviously received content item the received content item is deleted asshown at 714 and the process terminates on the end block If uponde-duping at 706, it is determined that the content item is an originalcontent item and does not duplicate information previously forwarded tothe user, the received item of content is scored by the user PE in orderto further determine its relevance to the user as shown at 708. In oneembodiment, the relevance of the content item is determined based on thecurrent user context. This determination based on the user's context caninclude a comparison of these features for the user to those for eachitem. By the way of illustration and not limitation, the current contextcan comprise time of day, day of week, month of year, and location. Theuser PE does this determination of relevance by noting the meta-data foreach item, when it is interacted with by the user, and then preferringitems with similar meta-data when the user is in a similar context. Forexample, if the user clicks on a weather forecast item in the morning,then weather forecasts will more likely be recommended in the morning inthe future. Similarly, clicking on a traffic alert while the user is atwork will lead to more traffic alerts at work. In other aspects,metadata associated with items such as identity of a sender or presenceof certain keywords can be considered in the determination of contextualrelevance. At 710, a determination of relevance is made based on thescore. If, it is determined that the content item is not relevant to theuser, the content item is deleted as shown at 714 and the processterminates on the end block without forwarding the content item to theuser's unseen items list. In particular, the score is compared with athreshold parameter and if the relevance score exceeds the thresholdparameter, the received item of content is determined to be relevant andis therefore added to the unseen items cache as shown at 712. In oneembodiment, an unseen list of items is maintained for each content type.For example, an unseen content items cache can be maintained for each ofemail type of content, news content type, FACEBOOK posts or other typesof contents based on the content streams that a user PE is configured toprocess. At 712, it is determined if more content items need to beprocessed in order to determine their relevance to the user. If yes, themethod loops back to step 704 wherein the next content item is selected.If no more content items exist, a list of relevant items is assembled asshown at 716 from the unseen items list and served to the user inaccordance with an embodiment as will be further detailed infra. Thus,the recommendations module 210 has two stages of filtering wherein thefeature PEs forward a first subset of received content items to the userPEs and the user PEs will added a subset of the received first subset ofcontent items to the unseen items lists. Additionally, therecommendations module 210 also includes a third stage of furtherfiltering wherein content items will be re-scored prior to beingforwarded to the user as will be detailed further infra. It may beappreciated that filtering by feature PEs and scoring by user PEs canoccur constantly in real-tine as new content flows into therecommendation module 210.

FIG. 8 shows a flowchart 800 illustrating an embodiment of a method offorwarding relevant content items to a user. The method begins at 802wherein a user request for fresh, unseen content is received. In oneembodiment, the user request can be received in the form of a ‘gimme’gesture. For example, the user can click a ‘gimme 5’ button 150 includedon the user interface indicating the user's desire to receive the topfive content items from the various content types pre-assembled in theunseen content items lists as detailed supra. It may be appreciated thatwhile the “gimme 5” gesture is based on the colloquial “Gimme 5!”, thenumber of content items served to the user in response to a user requestis only specified herein by the way of illustration and not limitationand that any number of content items can be served based, for example,on the determination of relevance and the availability of the contentitems to the user PE. Content consumption experiences typically give theuser little or no control over the rate at which content is served orrecommended to them. Therefore, in order to provide more control to theusers over the volume of content served to them, the “gimme” gesture canbe paired with an indicator to the user of how much content they aregoing to receive when they take the action. If no particular amount isspecified, the recommendations module 210 can default to a presetnumber. In a further embodiment, the default number of content piecesserved to a user can be determined for each user uniquely and can alsobe adjusted dynamically based on the rate at which the user consumescontent or the frequency of user interaction with the recommendationsmodule 210. When a “gimme” is executed, the user is served a number ofpieces of content in accordance with the expected number. In anembodiment, they are not served more content unless they execute the“gimme” gesture or request for content again.

At 804, the unseen items cache is accessed. The cache of unseen itemscomprises of those content items that are determined to be relevant tothe user upon scoring by the user PE 332. In an embodiment, the cachecomprises separate list of unseen items for each content type.Accordingly, the lists can comprise by the way of illustration and notlimitation, an unseen emails list that comprises, for example, the tenlatest, unseen emails that are determined to be relevant to the user, anunseen news items list which can comprise latest, relevant news itemswhich were not seen by the user, a list of latest, unseen, FACEBOOKposts, and a list of latest, unseen tweets from TWITTER. At 806, thecontent items are rescored in order to determine their relevance to theuser based on, for example, current user context in accordance withembodiments as described herein. At 808, the relevance score of thecontent items obtained at step 806 is compared with the relevancethreshold and a predetermined number of content items that still exceedthe relevance threshold are selected at 810. The selected content itemsthat exceed the relevance threshold are sorted by the temporal metadataassociated therewith at 812. In one embodiment, the temporal metadatacan be a time at which the particular content item was received. In oneembodiment, the temporal metadata can be a time at which a particularcontent item was published. The selected content items can either besorted from the latest to the oldest content item or conversely from theoldest to the latest content items based on, for example, userpreferences associated with different types of content items. Therecommendations module 210 can permit the user to select the sort orderso that the user may select different sort orders for different contenttypes. For example, for private data such as emails, the user can chooseto receive the oldest unseen relevant email first whereas for publicdata such as news item types, the user can choose to receive the latestunseen news item first.

At 814, the item mixture weights for different content types aredetermined. As described herein, the recommendations module 210 takes awide variety of content types as input and determines not only how toscore them but also how to ensure that there is a reasonable amount ofdiversity in the recommendations or content lists forwarded to the user.In an embodiment, the fudge factor F described supra with respect toexpression (4) is employed for scaling the scoring function based oncontent types. In addition, each major content type (e.g., News,FACEBOOK, TWITTER, Email) has a target percentage associated with it,which varies over time as the user interacts with the recommendationsmodule 210. By the way of illustration and not limitation, the contenttypes that can be included in the recommendations module 210 cancomprise: Email (from YAHOO! or any other email provider), FACEBOOKNewsfeed (personal), GOOGLE+ feeds (personal), Horoscope, LINKEDIN News,status updates, Local News, Local deals and ads, Local events, Localpoints of interest, Local traffic reports, Local weather and weatheralerts, RSS (Really Simple Syndication) feeds (user selected), RSS feeds(curated), Stock market News (from personal portfolio), Stock marketalerts (from personal portfolio), TWITTER accounts (curated), TWITTERstream (personal), YAHOO Answers (personal question and answers), YAHOOFresh (web content from trends on Twitter), YAHOO News (by category),YAHOO! News For You, YAHOO! News friend activity (aka Social Chrome),YAHOO! content sites (Shine, OMG, Green, etc.), YAHOO Groups (personal),photos from photo-sharing sites. When selecting which items torecommend, the user PE 322 walks through each content type (startingwith the one with the largest target), and tries to recommend enoughitems from that type that pass the relevance threshold so that the ratiois preserved. If it cannot recommend the requisite amount of aparticular content type, it then readjusts the target percentages of theremaining content types so as to maintain their relative ratios.Furthermore, as the user provides positive or negative feedback for anitem of a given content type, its percentage is adjusted upwards ordownwards by a small amount, so that for example, interacting positivelywith a given type will increase its prevalence in the items recommended.There are both minimum and maximum limits on the percentages, so no onetype comes to dominate the stream of recommended items.

At 816, a target number of items for each of the item types can beselected to forward to the user based on the item type weights and thenumber of new content items requested by the user as described herein.Based on the item weight associated with each content type, a ratio ofthe number of content items of each type to be included in the mixtureof item types forwarded to the user is determined. Again, as describedherein various content items including public and private data such asemails, new items, alerts, advertisements, social networking feeds andinput from other selected sources can be included in the content sent tothe user. Thus, if the target ratio to include a particular type ofcontent in the resultant items list to be forwarded to the user is less,there will be less number of that particular type of content items inthe resultant items list forwarded to the user. Conversely, if thetarget ratio to include a particular type of content in the resultantitems list to be forwarded to the user is higher, there will be more ofthat particular type of content items in the resultant items listforwarded to the user.

The resultant items list with the selected number of each item type isforwarded or transmitted to the user as a seen items list as shown at818. In an embodiment, it can happen that the number of new contentitems ‘X’ determined to be relevant to the user upon rescoring at 806 isless than the number of new content items ‘N’ requested by the user. Inthis case, either only the relevant new content items ‘X’ as requestedby the user are displayed to the user on a user device or a combinationof the ‘X’ relevant new content items and top ‘N-X’ content itemspreviously viewed by the user can be displayed to the user.

FIG. 9 is a schematic figure showing the processes executed by a user PE332 in accordance with one embodiment. The user PE 332 upon receiving avector representation of a content item D, scores the content item D forits relevance to the user in real time as shown at 952. In particular, amodified dot product of a vector representation 540 of the content itemD 502 is obtained in combination with the user vector 944 as shown at952. The resultant of the operation at 952 includes the user vector 944which represents the user in combination with user preferences includingbut not limited to keywords, categories and sources. Additionally,metadata related to current user context such as date/time/user locationobtained for example, via GPS (Global Positioning System) coordinatescellular, triangulation/positioning or IP (Internet Protocol) addresscan also be employed to determine relevance of the content item D 502 tothe current user context. If the received content item D 502 isdetermined to be relevant, it is added to one of the unseen item lists954. For example, if the item D 502 is a news item which is determinedto be relevant to the user it is added to the unseen news queue. Asshown in FIG. 9, each of the item types has a respective unseen itemsqueue in addition to being associated with a particular item mixtureweight. Thus, the news item type has a respective item mixture weightassociated therewith. When a request for content, for example, in theform of a ‘gimme’ gesture is received from the user as shown at 956, thevarious items from the different item queues 954 are rescored inaccordance with respective weights and combined as shown at 958. In thisinstance depicted in FIG. 9, content item D 502 is found to be relevantupon rescoring at 958 and accordingly, it is forwarded for display tothe user in the seen items queue 960.

If the user, upon receiving the content item D 502 in the seen queue960, clicks on it, the click event 962 is received by the user PE 332and used as a feedback to update item mixture weights in real time asshown at 964. In one embodiment, as the item D 502 was a news type item,the News Weight associated with the news content type is increased sothat subsequent seen item lists forwarded to the user will be biased toinclude a greater number of news type items. Conversely, if the user hadnot clicked on the content item D 502, such information is also receivedby the user PE 332 and employed to update the mixture weights 964 sothat less number of content items of the content type that remainedun-selected by the user are included in the subsequent seen lists thatare forwarded to the user.

Additionally, the feedback regarding the user selection of the contentitem D 502 is employed in updating the user vector in real time as shownat 966. Thus, actions that the user takes on items translate intoreal-time feedback using a standard technique called Rocchio feedback,wherein the item's vector, in this instance vector 540 is added orsubtracted from the user vector 944 after multiplying it by some weight.If the user clicks on the content item D 502, the user vector 944 iscombined with the representation of the content item D 502, theinformation regarding the click event, for example, the temporalmetadata associated with the click event, and the user preferences inorder to recalculate the user vector. Therefore, each time a userselects an item of content, the event generated by such selectionaffects the user model 3324 in real time. In an embodiment, theselection information of the user can also affect other user modelswhich are similar to the user model 3324 so that the content itemsforwarded to the other users are also affected in real-time. In anembodiment, in addition to being updated in real-time, the user vectorcan also be updated on a periodic basis, for example every twenty fourhours, as will be detailed further herein.

In addition to updating the user model 3324, the feedback regarding theuser click events can also be employed for content discovery. Forexample, any new terms that get added to a user's model 3324 as a resultof feedback immediately (in real-time) cause their user PEs tore-evaluate any item in the recommendations module 210 that contains oneof the new terms. This has the effect of making the recommendationsmodule 210 extremely responsive to the user's current interests, byrecommending related items to what the user has just read or otherwiseinteracted with. Furthermore, actions taken by individual users onpublicly available items are noted by the Item PE, which resends itselfto all interested users for rescoring, for example, via the users listassociated with the various feature PEs which include that particularitem in their list of items. And because the aggregate actions of otherusers of the recommendations module 210 are also used when scoring anitem, this can result in, for example, more popular items being morelikely to get recommended as they become more popular. Also, wheneverthe user model changes, items that have previously passed the thresholdfor recommendation are re-scored to ensure they still pass.

FIG. 10 is a schematic diagram that illustrates a procedure of updatingthe user PE on a periodic basis, for example, on a nightly basis. Asdiscussed herein, all category-level vectors for content categories thata user has selected are aggregated and this aggregate is assigned as theinitial user model. Accordingly, the user PE 332 includes contentcategories 1002 that are explicitly declared by the user for example, asshown in the user preferences 420, or are implicitly derived as being ofinterest to the user via various user actions and/or input from otherrelated users or users with similar interests. A few content categoriessuch as business, entertainment, health and world news categories areshown to be included in the user PE 332 by the way of illustration andnot limitation. On a periodic basis, for example, on a nightly basis,the features such as word/terms which are included in the categories1002 in addition the content items from the sources specified in theuser preferences 420 are folded or added to the user vector 944 as shownat 1004. By folding in the cold-start model or the latest version of thecategories included in the user PE 332 on an continuing basis, therecommendations module 210 ensures that content on new topics isrecommended to the user associated with the user PE 332, especially asnew topics are always being created.

In addition to being updated with the cold start category vectors on aperiodic basis, the user vector 944 is additionally aggregated withweighted user vectors over the previous thirty days as shown at 1006.Snapshots of the user model are obtained on a periodic basis (forexample, daily), and kept for some time into the past. When folding thecold-start model in, an aggregate of the user's model over the past(weighted exponentially for example, to emphasize recent user models) isalso folded in. In this way, long-term interests are kept in the user'smodel, so that even if they haven't interacted with an item with aparticular term recently, it won't completely drop out of their modelfrom lack of reinforcement. Accordingly, the user vectors of the user PE332 of the last thirty days weighted by days back are obtained andcombined with the current version of cold start category vectors inorder to obtain the updated user vector 944 as shown at 1008 and asgiven by Expression (1) supra. The recommendation module 210 recordswhat the user is interested in and exploits this information so thatmore content that is aligned to the user's interests is recommended.Saving the user vectors on a daily basis, and then rolling them backinto the next day's user vector, allows for incorporation of short-terminterests while also not forgetting long-term interests. This alsoensures that short-term interests (caused by spiking news events) do notoverwhelm a user vector. Therefore, the updated user vector 944 is acombination of the cold start category vectors of those categoriesselected for inclusion into the user PE 322 and the user vectors of theprevious thirty days as shown at 1008. Thus, user model for the next dayis:

-   -   alpha*decayed weighted sum of user model over last 30 days, plus    -   (1-alpha)*sum of category vectors from recent News articles.

In addition to being automatically updated in real time and on periodicbasis, the recommendations module 210 allows a user to manipulate theuser vector 944 to include keywords directly as shown at 1010 so thatthe user vector 944 can be configured to better represent the user.Users can explicitly add terms to their respective user models,effectively “whitelisting” these terms. Such terms are added to the uservector with a weight of 1. Items that contain these terms are alsoupweighted, so that terms on the whitelist are effectively standingsearches and content items comprising such terms will be weighed morefavorably in comparison to other results that do not comprise suchterms. Additionally, users have a “blacklist” of terms that they canedit, effectively removing these terms from their model, thus whilepresenting recommendations to the user, the content items comprising theblacklisted terms will be weighed unfavorably thereby greatly reducingthe probability that items with these terms will be recommended. In anembodiment, UI (user interface) of the recommendations module 210 canalso treat whitelist terms specially when rendering items forrecommendation. For example, by highlighting these terms as ones theuser has told the system are important to them. This makes it easy forthe user to see items about terms they are particularly interested in.The UI also has a mode in which terms from an item are recommended tothe user, who then has the option of adding them to their whitelist orblacklist. The recommended terms are selected from the correspondingitem, and reflect the most salient terms from that item, including forexample, a source of the content item, a location associated with thecontent item, or the author's name. Thus, the recommendations module 210can operate in a “training mode” suggesting material to the user thatcan be used to fine-tune the user model 3324.

FIG. 11A shows a flow chart 1150 illustrating an embodiment of a methodof initiating a user model in accordance with one embodiment. The methodcommences at 1102 wherein a user request to register with therecommendations module 210 is received. In response to the user request,the recommendations module 210 can request and obtain informationregarding the user as shown at 1104. This can include user's personalattributes such as age, sex, location and other demographic information,user's interests such as including but not limited to, keywords andentities. In addition, information regarding the sources from which theuser would like to receive the updated content recommendations is alsoreceived. At 1106, the user can be provided with a list of content itemcategories to choose from. For example, the content categories caninclude news categories such as world news, entertainment news and othernews categories. In one embodiment, the user can be presented with asuperset of all the content item categories available for user selectionwithin the recommendations module 210. In one embodiment, a subset ofthe available categories can be selected based on the informationprovided by the user at 1104. At 1108, the user's selections ofcategories are received. At 1112, various avatars are provided for userselection. In one embodiment, the avatars provided for user selectioncan be based on the content categories selected by the user so that theavatar personifies the user model that will be generated to reflectuser's interests. At 1114, user selection of an avatar is received andthe user model is generated at 1116 based on the information collectedfrom the user at various steps of the procedure described supra. In oneembodiment the category vectors, keywords, entities and other attributeinformation is aggregated to obtain the user model. For example, vectorsof categories that the user is interested in are added and within thesecategories, particular keywords that the user blacklists can besubtracted to obtain a resultant user vector that models the user.Hence, the recommendations module 210 models the user in similar fashionas items which makes users and items somewhat interchangeable, aproperty which is exploited in a number of different ways as detailedherein.

FIG. 11B shows a flowchart 1100 illustrating an embodiment of a methodof updating a user vector on a periodic basis. The method begins at 1110wherein the cold start vectors of the categories included in the user PE322 are selected. They are combined as shown at 1120. Such explorationallows a user to discover content on topics they may not have previouslyseen. This can include trending topics. In one embodiment, a snapshot ofthe user vector just prior to being updated on a periodic basis can beobtained and stored. Such user vectors of the last N days can beobtained as shown at 1130. In one embodiment N is a natural number whichcan take one of the values 1, 2, 3 . . . . . In an embodiment, the valueof N can be determined dynamically and can vary based on, for example,the user attributes. In an embodiment, N=30. Each of the user vectors iscombined with a weighing factor that biases the combination of the uservectors so that the latest user vector carries the greatest weight andthe oldest user vector carries the least weight as shown at 1140 andsuch weighted user vectors of the previous thirty days are combined asshown at 1150. At 1160, the combination of the cold start categoryvectors obtained at step 1120 is aggregated with the combination of theweighted user vectors obtained at 1150 and an updated user vector isgenerated as shown at 1170. In an embodiment, whenever the user vectorchanges, items that have previously passed the threshold forrecommendation are re-scored to ensure that they still satisfy therelevance threshold for the user in view of the updated user vector.Generally items that have passed the relevance threshold and are not yettransmitted to the user are stored in unseen items list as describedherein. Therefore, the items in the unseen items list are rescored at1180 in order to determine that they are relevant to the updated uservector. Those that satisfy the relevance threshold are retained in theunseen items cache whereas those that fail to fulfill the relevancethreshold are discarded as shown at 1190.

FIG. 12A shows a flowchart 1200 illustrating an embodiment of a methodof updating user recommendations based on user feedback. The methodbegins at 1202 wherein the user feedback is received. The user feedbackcan comprise a user click on a content item in order to view the contentitem. It can also comprise additional user actions such as but notlimited to, forwarding the content item to other contacts, voting for oragainst the content item or providing comments in response to thecontent item. Upon receiving the feedback, new terms associated withthat particular content item can be obtained and the user model can beupdated as shown at 1204. The new terms can be obtained, for example,from the item PE associated with the content item. New content itemsassociated with the newly added terms are obtained at 1206. The usermodel can obtain the new content items, for example, via the list ofcontent items comprised within the feature PEs associated with the newlyadded terms. The content items received at step 1206 are scored at 1208in order to determine their relevance to the user who is providing thefeedback in accordance with embodiments described herein. At 1210, thedetermined relevance is compared to the relevance threshold for theuser. If the content item satisfies the threshold relevance at 1210, itis added to the unseen items list as shown at 1212 to be furtherprocessed upon receiving a user request in accordance with embodimentsdescribed herein. If the content item does not satisfy the relevancethreshold at 1210, it is discarded as shown at 1214 and the procedureterminates on the end block. It can be appreciated that theincorporation of the user feedback into the recommendations provided tothe user can occur in real-time. Thus, the process described in FIG. 12Acan occur in real-time so that the list of unseen items varies from oneuser request or one user click of the ‘gimme’ button to the next click.

FIG. 12B shows a flowchart 1250 illustrating an embodiment of a methodof updating user recommendations based on user feedback. The methodbegins at 1252 wherein the user feedback is received. Upon receiving thefeedback, new terms associated with that particular content item can beobtained and the user model is updated with the new terms as shown at1254. The new terms can be obtained, for example, from the item PEassociated with the content item. The content items added to the unseenitems lists in the respective user PE are restored to determine anupdated relevance score in view of the updated user model as shown at1256. At 1258, the newly determined relevance score is compared to therelevance threshold for the user. If the content item satisfies therelevance threshold at 1258, it is remains in the unseen items list asshown at 1260 to be further processed upon receiving a user request inaccordance with embodiments described herein. If the content item doesnot satisfy the relevance threshold at 1258, it is discarded as shown at1262 and the procedure terminates on the end block. It can beappreciated that the incorporation of the user feedback into therecommendations provided to the user can occur in real-time.

In an embodiment, user PEs of different users can send messages to eachother. A direct comparison of two user models using a similarity measuredescribe herein can result in a recommendation that users connect toeach other, either on one of the client services (TWITTER, FACEBOOKetc.) or within the recommendations module 210. If the users alreadyhave established a connection within the recommendations module 210,user A could “follow” user B, with the result being that any publiclyavailable recommendations that user B's User PE recommends are also sentto user A's User PE for evaluation and possible recommendation.

FIG. 13 is a flow chart 1300 illustrating an embodiment of a method ofproviding user recommendations based on similarity between user models.The method begins at 1310 wherein two user models are compared. Thiscomparison can occur, for example, when a user selects certain categoryvectors to be included into the user model. Based on such selectionand/or other user attributes such as, location, age, sex, profession forexample, other users who have similar category vectors in their usermodels can be selected for comparison. At 1320, it is determined if thetwo user models are similar based on embodiments described herein. In anembodiment, techniques such as cosine similarity measure can be employedto make the similarity determination at 1320. If the two user models arenot similar the procedure terminates on the end block. If it isdetermined at 1320 that there is sufficient similarity between the twouser models, it is determined at 1330 if the two users are connectedwithin the recommendations module 210. If the two users are notconnected, then a message can be sent at 1340 suggesting that the usersconnect within the recommendations module 210 or any other socialnetwork of the users' choice and the procedure terminates on the endblock. If it is determined at 1330 that the users are connected, thecontent item recommendations made by the users to their social contactsor the content items indicated as being of interest to the users areexchanged at 1350. Therefore, users with similar user models can besources of content recommendations within the recommendations module210.

FIG. 14 is a flow chart 1400 illustrating an embodiment of a method ofupdating user recommendations in real time in response to received userfeedback. As described supra, the user PE 332 receives events associatedwith user actions, such as user clicks on particular content itemsforwarded to them. In one embodiment, the user PE 332 can incorporatefeedback from such user actions and update content recommendationsprovided to the user in real-time. The method commences at 1402 with theuser PE receiving the feedback, for example, a user click on aparticular content item. At 1404, the type of content item selected bythe user is determined and the item type mixture weight associated withthat content type is updated as shown at 1406. In one embodiment, theitem type mixture weight of the particular type of content selected bythe user is increased and correspondingly, the item type mixture weightsof other content types can be decreased so that the ratio the contenttype selected by the user in the overall content transmitted to the useris increased. At 1408, content items from the unseen items lists arere-selected based on the updated item type mixture weights. An updatedcontent stream is transmitted to the user as shown at 1410. In anembodiment, the procedure of receiving the user feedback and updatingthe content recommendations can occur in real-time so that by justclicking or selecting a particular content item from the recommendedcontents can cause the content recommendations to be updated.

In the context of the recommendations system 100, an avatar is agraphical image of a person or other animated or anthropomorphizedobject that represents the user model 3324. In an embodiment it acts asa bridge between the user and their otherwise opaque model 3324. In anembodiment, the avatar is selected by the user while initiallyconfiguring the recommendations module 210 and a selection of the avatarcan be factored into building the user model 3324. FIG. 15A shows anembodiment of a methodology 1500 that details the procedure ofgenerating the avatar. As detailed supra, when a user initiallyregisters with the recommendations module 210, a set of contentcategories is presented to the user for selection. Based on the userselection of content categories, a set of avatars can be presented foruser selection as shown at 1504. In an embodiment, the set of avatarspresented to the user can be a subset of all the avatars availablewithin the recommendations module 210 for user selection. At 1506, auser selection of one of the avatars is received. In an embodiment, theavatar can be employed to allow the user to interact with the user model3324 thereby removing at least some of the veil of obscurity about themodel 3324. In fact, the avatar is employed within the recommendationsmodule for providing help to the user with various tasks as will bedetailed further herein. At 1508, the user selected avatar is factoredinto the user model 3324 either while generating the user model 3324 orwhile updating the user model 3324. Therefore, the avatar is not onlyaffected by the user model but also affects the user model 3324 inaccordance with the embodiments described herein.

FIG. 15B shows an embodiment 1550 of methodology executable by aprocessor of, for example, a client device 202/204 generating an avatarin accordance with one embodiment. The method begins at 1552 wherein auser request for content recommendations from the recommendations moduleis transmitted by the processor. At 1554, a plurality of contentcategories for user selection are received by the processor from therecommendations module 210. At 1556 at least a subset of the categoriesselected by the user are received and at 1558 a subset of avatars forselection by the user, are displayed wherein the subset of avatars areindicative of the user selected categories. A user selection of anavatar from the subset of avatars is received at 1560 and therebycommunication from a user model associated with the avatar to the uservia the avatar is facilitated by the processor as shown at 1562, forexample, by displaying messages from the user model on a displayassociated with the client device 202/204.

FIG. 16 depicts an embodiment, wherein the avatar 1602 is employed toact as a help agent, providing hints on how to best use and customizevarious aspects of the recommendations module 210, in the form ofmessages. As described supra, the user model 3324 is able to process theevents generated from the user actions. As the avatar 1602 is tied tothe user model 3324, the avatar can provide contextual help in real-timeby monitoring user actions. In FIG. 16, the avatar 1602 generates threedifferent messages 1610, 1620 and 1630 based on the context and useractions. In fact, based at least one the messages 1610, 1620 and 1630different buttons 1640 can be displayed appropriately to the user. Forexample, at 1610, the avatar advises the user of the whitelistfunctionality available within the recommendations module 210 whereincertain terms can be added to the user model as standing searcheswherein content comprising the whitelisted terms is retrieved andpresented to the user as it is received by the recommendations module210. Accordingly, a graduation cap button 1650 is displayed which allowsthe user to begin training the user model 3324 as will be detailedfurther infra. In response to the message 1610, the user has whitelistedsome terms, and content which is retrieved in response to such standingsearch queries has been presented. At 1620, the avatar advises the userof the highlighted terms in the content being the whitelisted terms orstanding search queries. It further advises on how to change the termsif they are not to the user's liking. Further at 1630, if the user isreceiving too much content associated with the standing search queries,the user model 3324 is able to identify such a situation and sends acontext sensitive or contextual message via the avatar 1602 about theremoval of the whitelisted terms. Based on the user response to thecontextual messages, the standing search queries can be updated.Therefore, the user can add/remove and/or edit the standing searchesinstituted by the user model 3324 via exchange of contextual messageswith the avatar 1602. Thus, the avatar 1602 is able to personify theuser model 3324 to the users and is able to act as an interface throughwhich the user model 3324 is able to communicate with the user.

As mentioned above, a “training mode” recommends terms for the user towhitelist or blacklist—the avatar is featured in this mode in order tomake it clear that the user's actions are directly affecting theirmodel. FIG. 17 is depicts a schematic diagram of an avatar beingemployed when the user model 3324 is operating in a training mode. Inparticular, FIG. 17 shows a message from the avatar 1702 suggesting theaddition of terms to standing searches. Accordingly, a training modebutton 1650 is displayed. When the user clicks on the “training mode”button 1650, a pop-up 1710 appears that allows the user to take thesuggested actions to train their user model 3324. As seen from FIG. 17,the pop up allows the user to add new terms or remove existing termsfrom the user model 3324 by clicking the + or − buttons respectively. Itcan be appreciated that the pop-up 1710 is shown only for illustrationand is by no means limiting. In different embodiments, othermethodologies or other user interface elements can be employed toupdate/edit the user model 3324.

An avatar can have different levels of “health” (which are rendered inthe avatar's image, as well as elsewhere) depending on the user'sengagement level with the recommendations module 210, as well as theirlevel of customization of the recommendations module 210. FIG. 18a showsan avatar 1802 from a subset of avatars presented for user selection inaccordance with embodiments described herein. In addition to urging theuser to select it for representing the user model 3324 via the “Pickme!” button, the avatar also informs the user about how it can becomehealthy with time via the user interaction and the various levels ofhealth 1804 it has associated therewith in accordance with embodimentsdescribed herein. In one embodiment, the health level of the avatar isindicated via its color. The avatar 1802 can be initially colored inshades of black or white and/or grey indicating a neutral state or astate wherein its health level is not yet set as the user model 3324 isstill being configured. In an embodiment, different health levels of theavatar 1802 can correspond to the different colors of the visiblespectrum for example, with red being associated with lowest health andeach color in the spectrum being associated with a progressivelyincreasing health of the avatar 1802. Thus, for a user who is justbeginning to use the recommendations module 210 and whose user model3324 is fairly new, the avatar can be colored red. However, it cangradually change in intensity and eventually transform into a differentcolor with increasing time and frequency of the user interaction withthe user model 3324.

In another embodiment, the avatar 1802 health can be indicated via onlya single color but the intensity of the color can indicate the health ofthe avatar 1802. For example, a healthy avatar will be vividly coloredindicating a user who interacts frequently with the recommendationsmodule 210 and is actively involved in developing the respective usermodel. Another user who does not interact as frequently with therecommendations module 210 and seldom provides feedback may have anavatar of the same color as the frequent user, but the occasional user'savatar can be associated with a dull shade which is less intense thanthat of the frequent user.

FIG. 18b shows various facial expressions associated with the avatar1850 by the way of illustration and not limitation. In an embodiment,the different facial expressions of the avatar can be associated withthe various levels of health associated with the corresponding usermodel. The sad expressions 1860 and 1870 of the avatar indicate that itis not healthy. In one embodiment, the health level/facial expressioncan indicate that the corresponding user is an occasional user of therecommendations module 210 who infrequently provides feedback or whodoes not have enough number of accounts/content streams associated withthe recommendations module 210 and thereby is not providing enoughinformation about himself/herself to the user model. In one embodiment,this can be the expression of an avatar at the initial stages of theuser interaction with the recommendations module 210. But eventually asthe user interacts more frequently with the recommendations module 210and gets more involved with customizing the user model 3324 by providingexplicit input or implicit feedback via various actions, the expressionof the avatar can brighten up to display a more cheerful countenance asshown at 1880 and finally at 1890 the expression indicates a veryhealthy avatar associated with or personifying a user model whose useris involved with the recommendations module 210 and provides plenty ofinformation for customizing the user model.

In one embodiment, the avatar can likewise reflect the emotionalsentiment of the content. In this case, the avatar's facial expression,or other way of rendering the avatar such as, for example, differentcolors, or other visual treatment can indicate sentiment. FIG. 19 showsa depiction of an embodiment wherein various expressions of the avatar1850 reflect various emotions in response to different content. Asmentioned supra, different types of content is rendered and theexpression of the avatar associated with a particular content type willdepend on the exact content in the content item and in an embodiment,the expression of the avatar for a particular content item can beselected based on the user's likes/dislikes. In an embodiment, onlycontent item associated with public data sources, such as news sources,or blogs that are freely available will have a related avatar displayedtherewith whereas private content can be displayed without the avatar.In an embodiment, the avatar can reflect the user's likely reaction to aparticular news item. In this context, it can be appreciated that whiletwo users who select the same avatar may generally react similarly to aparticular content item, it is also possible that they may reactdifferently to different content items. Accordingly, their avatars canreflect different expressions for the same content item indicating thesubtle differences that may exist in their user models. In anembodiment, the expressions of the avatars can be selected by processingthe content via natural language processing techniques, analyzing thecontent with sentiment analysis tools and/or analyzing the content incontext of the user information available with the user model 3324 andidentifying a graphic of the avatar from a predetermined set of images,with the appropriate expression.

As a representative of the user's recommendation model 3324, the avatarcould travel with the user across the web. In this context, arepresentation of the avatar can communicate to the user where and whencontent is being personalized or not. FIG. 20A depicts an embodimentwherein the avatar 1850 is rendered on an online content being viewed bya user. The type of rendering can be a visual indication of whether ornot the associated content has been personalized. In an embodiment, afull-color and/or smiling avatar as shown in FIG. 20A indicates thecontent is personalized.

FIG. 20B depicts an embodiment wherein the avatar 1850 is rendered in anonline content item being viewed by a user. In this embodiment, theavatar 1850 appears grayed-out and/or with a neutral expressionindicating that the content is not personalized. In an embodiment, theuser can interact with (e.g., click) the avatar 1850 to toggle betweenpersonalized and non-personalized content thereby changing the renditionof the avatar 1850 with the associated content.

As shown in the example of FIG. 21, internal architecture of a computingdevice 2100 includes one or more processing units (also referred toherein as CPUs) 2112, which interface with at least one computer bus2102. Also interfacing with computer bus 2102 are persistent storagemedium/media 2106, network interface 2114, memory 2104, e.g., randomaccess memory (RAM), run-time transient memory, read only memory (ROM),etc., media disk drive interface 2108, an interface 2120 for a drivethat can read and/or write to media including removable media such asfloppy, CD-ROM, DVD, etc., media, display interface 2110 as interfacefor a monitor or other display device, keyboard interface 2116 asinterface for a keyboard, pointing device interface 2118 as an interfacefor a mouse or other pointing device, and miscellaneous other interfaces2122 not shown individually, such as parallel and serial portinterfaces, a universal serial bus (USB) interface, and the like.

Memory 2104 interfaces with computer bus 2102 so as to provideinformation stored in memory 2104 to CPU 2112 during execution ofsoftware programs such as an operating system, application programs,device drivers, and software modules that comprise program code, and/orcomputer-executable process steps, incorporating functionality describedherein, e.g., one or more of process flows described herein. CPU 2112first loads computer-executable process steps from storage, e.g., memory2104, storage medium/media 2106, removable media drive, and/or otherstorage device. CPU 2112 can then execute the stored process steps inorder to execute the loaded computer-executable process steps. Storeddata, e.g., data stored by a storage device, can be accessed by CPU 2112during the execution of computer-executable process steps.

Persistent storage medium/media 2106 is a computer readable storagemedium(s) that can be used to store software and data, e.g., anoperating system and one or more application programs. Persistentstorage medium/media 2106 can also be used to store device drivers, suchas one or more of a digital camera driver, monitor driver, printerdriver, scanner driver, or other device drivers, web pages, contentfiles, playlists and other files. Persistent storage medium/media 2106can further include program modules and data files used to implement oneor more embodiments of the present disclosure.

FIG. 22 is a schematic diagram illustrating a client deviceimplementation of a computing device in accordance with embodiments ofthe present disclosure. A client device 2200 may include a computingdevice capable of sending or receiving signals, such as via a wired or awireless network, and capable of running application software or “apps”.A client device may, for example, include a desktop computer or aportable device, such as a cellular telephone, a smart phone, a displaypager, a radio frequency (RF) device, an infrared (IR) device, aPersonal Digital Assistant (PDA), a handheld computer, a tabletcomputer, a laptop computer, a set top box, a wearable computer, anintegrated device combining various features, such as features of theforgoing devices, or the like.

A client device may vary in terms of capabilities or features. Theclient device can include standard components such as a CPU 2202, powersupply 2228, a memory 2218, ROM 2220, BIOS 2222, network interface(s)2230, audio interface 2232, display 2234, keypad 2236, illuminator 2238,I/O interface 2240. Claimed subject matter is intended to cover a widerange of potential variations. For example, the keypad 2236 of a cellphone may include a numeric keypad or a display 2234 of limitedfunctionality, such as a monochrome liquid crystal display (LCD) fordisplaying text. In contrast, however, as another example, a web-enabledclient device 2200 may include one or more physical or virtual keyboards2236, mass storage, one or more accelerometers, one or more gyroscopes,global positioning system (GPS) 2224 or other location identifying typecapability, Haptic interface 2242, or a display with a high degree offunctionality, such as a touch-sensitive color 2D or 3D display, forexample. The memory 2218 can include Random Access Memory 2204 includingan area for data storage 2208.

A client device may include or may execute a variety of operatingsystems 2206, including a personal computer operating system, such as aWindows, iOS or Linux, or a mobile operating system, such as iOS,Android, or Windows Mobile, or the like. A client device 2200 mayinclude or may execute a variety of possible applications 2210, such asa client software application 2214 enabling communication with otherdevices, such as communicating one or more messages such as via email,short message service (SMS), or multimedia message service (MMS),including via a network, such as a social network, including, forexample, Facebook, LinkedIn, Twitter, Flickr, or Google+, to provideonly a few possible examples. A client device 2200 may also include orexecute an application to communicate content, such as, for example,textual content, multimedia content, or the like. A client device 2200may also include or execute an application 2212 to perform a variety ofpossible tasks, such as browsing, searching, playing various forms ofcontent, including locally stored or streamed video, or games (such asfantasy sports leagues). The foregoing is provided to illustrate thatclaimed subject matter is intended to include a wide range of possiblefeatures or capabilities.

For the purposes of this disclosure a computer readable medium storescomputer data, which data can include computer program code that isexecutable by a computer, in machine readable form. By way of example,and not limitation, a computer readable medium may comprise computerreadable storage media, for tangible or fixed storage of data, orcommunication media for transient interpretation of code-containingsignals. Computer readable storage media, as used herein, refers tophysical or tangible storage (as opposed to signals) and includeswithout limitation volatile and non-volatile, removable andnon-removable media implemented in any method or technology for thetangible storage of information such as computer-readable instructions,data structures, program modules or other data. Computer readablestorage media includes, but is not limited to, RAM, ROM, EPROM, EEPROM,flash memory or other solid state memory technology, CD-ROM, DVD, orother optical storage, magnetic cassettes, magnetic tape, magnetic diskstorage or other magnetic storage devices, or any other physical ormaterial medium which can be used to tangibly store the desiredinformation or data or instructions and which can be accessed by acomputer or processor.

For the purposes of this disclosure a module is a software, hardware, orfirmware (or combinations thereof) system, process or functionality, orcomponent thereof, that performs or facilitates the processes, features,and/or functions described herein (with or without human interaction oraugmentation). A module can include sub-modules. Software components ofa module may be stored on a computer readable medium. Modules may beintegral to one or more servers, or be loaded and executed by one ormore servers. One or more modules may be grouped into an engine or anapplication.

Those skilled in the art will recognize that the methods and systems ofthe present disclosure may be implemented in many manners and as suchare not to be limited by the foregoing exemplary embodiments andexamples. In other words, functional elements being performed by singleor multiple components, in various combinations of hardware and softwareor firmware, and individual functions, may be distributed among softwareapplications at either the client or server or both. In this regard, anynumber of the features of the different embodiments described herein maybe combined into single or multiple embodiments, and alternateembodiments having fewer than, or more than, all of the featuresdescribed herein are possible. Functionality may also be, in whole or inpart, distributed among multiple components, in manners now known or tobecome known. Thus, myriad software/hardware/firmware combinations arepossible in achieving the functions, features, interfaces andpreferences described herein. Moreover, the scope of the presentdisclosure covers conventionally known manners for carrying out thedescribed features and functions and interfaces, as well as thosevariations and modifications that may be made to the hardware orsoftware or firmware components described herein as would be understoodby those skilled in the art now and hereafter.

While the system and method have been described in terms of one or moreembodiments, it is to be understood that the disclosure need not belimited to the disclosed embodiments. It is intended to cover variousmodifications and similar arrangements included within the spirit andscope of the claims, the scope of which should be accorded the broadestinterpretation so as to encompass all such modifications and similarstructures. The present disclosure includes any and all embodiments ofthe following claims.

What is claimed is:
 1. A method comprising: receiving, at a processor, aplurality of content items obtained from content streams of differentcontent types to be forwarded to at least one user, said receivingcomprising modeling each of the content items into a feature vectorrepresentation, each feature vector representation comprising data onthe respective vector indicating a content item's source, a contentitem's features and category; scoring, by the processor, the pluralityof content items based on a user model, said user model comprising avector representation of dynamically updated user information andcategorical information associated with interests of the user, saidscoring comprising comparing the vector representation of the user modelto each content item's vector and determining a score for each contentitem based on how similar each content item's vector corresponds to theuser model vector; adding, by the processor, at least a subset of thescored content items to respective lists of unseen content items for theuser based at least on respective scores; receiving, by the processor, arequest from the user for relevant unseen content items, said receivingcausing the processor to update said user model vector based on therequest, said updating causing the user information and categoricalinformation to be updated based on reference of the relevant unseencontent items in the request; rescoring, by the processor, the contentitems added to the lists of unseen content items based on the updateduser model, said rescoring comprising comparing the vectorrepresentations of the added content items to the user model andupdating the score for each added content item; determining, by theprocessor in response to the user request, a ratio of a number of thecontent items of each content type within the lists unseen contentitems, said determination comprising identifying a type of each contentitem included in each list and, based on each content item's score,determining said ratio that indicates which content item types havehigher scores; determining, by the processor, an item type mixtureweight for each content type, said item type mixture weight directlycorresponding to the determined ratio for each content type, wherein avalue of the item type mixture weight of one content item type isadjusted to balance out a value of the item type mixture weight ofanother content item type; selecting, by the processor from the subsetof content items, a target number of content items of each content typefor inclusion into the top scoring content items based on the determineditem type mixture weights; and forwarding, by the processor, theselected top scoring content items to the user in response to the userrequest for the relevant items.
 2. The method of claim 1, wherein thecontent streams of different content types are received from multiplecontent sources.
 3. The method of claim 1, further comprising:obtaining, by the processor, attributes of the received content items;identifying, by the processor, the content items to be forwarded to theuser based on the attributes.
 4. The method of claim 3, furthercomprising: forwarding, by the processor, the content items to the usermodel associated with the user upon identifying the attributes.
 5. Themethod of claim 1, wherein the user model comprises categoriesdetermined to be relevant to the user.
 6. The method of claim 1, whereinthe top scoring content items are forwarded as a list of seen items inreal-time.
 7. The method of claim 1, further comprising: comparing, bythe processor, a relevance score of the rescored content items with arelevance threshold in order to determine the top scoring content items.8. The method of claim 1, further comprising: sorting, by the processor,the top scoring content items based on respective temporal metadataassociated therewith.
 9. The method of claim 1, further comprising,receiving, by the processor, a user selection of a content item from theselected top scoring content items; recalculating, by the processor,relevance of the user selected content item to user models of otherusers on receiving the user selection.
 10. The method of claim 9,further comprising: modifying, by the processor, the item type mixtureweight of an item type selected by the user upon receiving the userselection.
 11. The method of claim 10, further comprising: determining,by the processor, an updated target number of content items of each itemtype to be forwarded to the user based on the modified weight of theitem type selected by the user.
 12. The method of claim 11, furthercomprising: reselecting, by the processor, the predetermined number oftop scoring content items from the subset of content items based on theupdated target number of content items; forwarding, by the processor,the reselected top scoring content items to the user in response to theuser selection.
 13. A computing device comprising: a processor; anon-transitory computer-readable storage medium for tangibly storingthereon program logic for execution by the processor, the program logiccomprising: logic executed by the processor for receiving a plurality ofcontent items obtained from content streams of different content typesto be forwarded to at least one user, said receiving comprising modelingeach of the content items into a feature vector representation, eachfeature vector representation comprising data on the respective vectorindicating a content item's source, a content item's features andcategory; logic executed by the processor for scoring the plurality ofcontent items based on a user model, said user model comprising a vectorrepresentation of dynamically updated user information and categoricalinformation associated with interests of the user, said scoringcomprising comparing the vector representation of the user model to eachcontent item's vector and determining a score for each content itembased on how similar each content item's vector corresponds to the usermodel vector; logic executed by the processor for adding at least asubset of the scored content items to respective lists of unseen contentitems for the user based at least on respective scores; logic executedby the processor for receiving a request from the user for relevantunseen content items, said receiving causing the processor to updatesaid user model vector based on the request, said updating causing theuser information and categorical information to be updated based onreference of the relevant unseen content items in the request; logicexecuted by the processor for rescoring the content items added to thelists of unseen content items based on the updated user model, saidrescoring comprising comparing the vector representations of the addedcontent items to the user model and updating the score for each addedcontent item; logic executed by the processor for determining, inresponse to the user request, a ratio of a number of the content itemsof each content type within the lists unseen content items, saiddetermination comprising identifying a type of each content itemincluded in each list and, based on each content item's score,determining said ratio that indicates which content item types havehigher scores; logic executed by the processor for determining an itemtype mixture weight for each content type, said item type mixture weightdirectly corresponding to the determined ratio for each content type,wherein a value of the item type mixture weight of one content item typeis adjusted to balance out a value of the item type mixture weight ofanother content item type; logic executed by the processor forselecting, from the subset of content items, a target number of contentitems of each content type for inclusion into the top scoring contentitems based on the determined item type mixture weights; and logicexecuted by the processor for forwarding the selected top scoringcontent items to the user in response to the user request for therelevant items.
 14. The device of claim 13, the content item receivinglogic further comprising: logic for receiving the content streams ofdifferent content types from multiple content sources.
 15. The device ofclaim 14, further comprising: obtaining logic, executed by theprocessor, for obtaining attributes of the received content items;identifying logic, executed by the processor, for identifying thecontent items to be forwarded to the user based on the attributes. 16.The device of claim 15, further comprising: forwarding logic, executedby the processor, for forwarding the content items to a user modelassociated with the user upon identifying the attributes.
 17. The deviceof claim 16, wherein the user model comprises categories determined tobe relevant to the user.
 18. The device of claim 13, further comprising,a list of seen items comprising the top scoring content items forwardedto the user.
 19. The device of claim 13, further comprising: comparinglogic, executed by the processor, for comparing a relevance score of therescored content items with a relevance threshold in order to determinethe top scoring content items.
 20. The device of claim 13, furthercomprising: advertising selection logic, executed by the processor, forselecting context sensitive advertisements based on the user model. 21.A non-transitory computer readable storage medium, having tangiblystored thereon, computer-executable instructions that when executed by aprocessor of a computing device, cause the processor to perform a methodcomprising: receiving, at the processor, a plurality of content itemsobtained from content streams of different content types to be forwardedto at least one user, said receiving comprising modeling each of thecontent items into a feature vector representation, each feature vectorrepresentation comprising data on the respective vector indicating acontent item's source, a content item's features and category; scoring,by the processor, the plurality of content items based on a user model,said user model comprising a vector representation of dynamicallyupdated user information and categorical information associated withinterests of the user, said scoring comprising comparing the vectorrepresentation of the user model to each content item's vector anddetermining a score for each content item based on how similar eachcontent item's vector corresponds to the user model vector; adding, bythe processor, at least a subset of the scored content items torespective lists of unseen content items for the user based at least onrespective scores; receiving, by the processor, a request from the userfor relevant unseen content items, said receiving causing the processorto update said user model vector based on the request, said updatingcausing the user information and categorical information to be updatedbased on reference of the relevant unseen content items in the request;rescoring, by the processor, the content items added to the lists ofunseen content items based on the updated user model, said rescoringcomprising comparing the vector representations of the added contentitems to the user model and updating the score for each added contentitem; determining, by the processor in response to the user request, aratio of a number of the content items of each content type within thelists unseen content items, said determination comprising identifying atype of each content item included in each list and, based on eachcontent item's score, determining said ratio that indicates whichcontent item types have higher scores; determining, by the processor, anitem type mixture weight for each content type, said item type mixtureweight directly corresponding to the determined ratio for each contenttype, wherein a value of the item type mixture weight of one contentitem type is adjusted to balance out a value of the item type mixtureweight of another content item type; selecting, by the processor fromthe subset of content items, a target number of content items of eachcontent type for inclusion into the top scoring content items based onthe determined item type mixture weights; and forwarding, by theprocessor, the selected top scoring content items to the user inresponse to the user request for the relevant items.